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Abstract 


In  this  research  a  new  approach  to  the  control  of  systems  represented  by  stochastic  differential 
equations  (SDKs)  is  developed  in  which  stochastic  control  is  viewed  as  deterministic  control  with  a 
particular  form  of  constraint  structure.  Specifically,  the  characteristic  “non-ant ici pal ivity”  property  of 
the  control  processes  is  formulated  a  an  equality  constraint  on  the  set  of  possibly  ant i< apatite 
processes.  The  optimal  non-anticipative  control  is  then  recovered  by  minimizing,  over  the  class  of 
possibly  anticipating  processes,  a  cost  function  modified  by  the  inclusion  of  a  Lagrange  multiplier  term 
to  enforce  the  nonanticipativity  constraint.  This  unconstrained  minimization  is  carried  out 
“pathwise"  i.e.  separately  for  each  value  of  the  random  parameter  anti  hence  ret  luces  to  a 
parametrized  family  of  deterministic  optimal  control  problems. 

Solutions  of  the  controlled  SDKs  with  anticipative  controls  are  defined  by  a  decomposition 
method.  It  is  shown  that  the  value  function  of  the  control  problem  is  the  unique  global  solution  of  a 
robust  equation  (random  partial  differential  equation)  associated  to  a  linear  backward  llamilton- 
Jacobi- Heilman  stochastic  partial  differential  equation  (HJR  SPDE).  This  appears  as  limiting  SI’DK  for 
a  sequence  of  random  I1JH  DDK's  when  linear  interpolation  approximation  of  the  Wiener  process  i- 
used.  Our  approach  extends  the  Wong-Zakai  type  results  [20]  from  SDK  to  the  stochastic  dynamic 
programming  equation  by  showing  how  this  arises  as  average  of  the  limit  of  a  sequence  of  deterministic 
dynamic  programming  equations.  The  stochastic  characteristic  method  of  Ktmita  [l.'l]  is  Used  to 
represent  the  value  function.  by  choosing  the  Lagrange  multiplier  equal  to  its  noiiaiiiicipalne 
constraint  value  the  usual  stochastic  (uonantiripalive)  optimal  control  and  optimal  cost  are  recovered. 
The  anticipative  optimal  control  problem  is  formulated  and  solved  by  almost  sure  deterministic 
optimal  control.  We  obtain  a  DDK  for  the  "cost  of  perfect  information”:  the  difference  between  the 
cost  function  of  the  nonanticipative  control  problem  and  the  cost  of  the  anticipative  problem  which 
satisfies  a  nonlinear  backward  HJB  SPDE.  Poisson  bracket  conditions  are  found  ensuring  this  has  a 
global  solution.  The  cost  of  perfect  information  is  shown  to  be  zero  when  a  Lagrangiati  submanifold  i-> 
invariant  for  the  stochastic  characteristics.  I.QCJ  and  a  nonlinear  anticipative  control  problem  ate 
considered  as  examples  in  this  framework  . 


KEYWORDS  :  stochastic  control,  optimal  control,  dynamic  programming,  cost  of  information, 
stochastic  partial  differential  equations,  stochastic  flows,  anticipative  stochastic  differential  equations 
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0  :  Introduction  and  presentation  of  the  report 


At  first  sight  it  appears  that  stochastic  control  is  simply  a  generalization  of  deterministic  control.  One 
sees  this  for  example  in  the  familiar  LQCi  problem:  given  the  solution  to  the  stochastic  linear  regulator 
problem  (with  white  noise  perturbations)  one  obtains  the  solution  for  the  deterministic  case  simply  in 
setting  the  noise  mean  and  covariance  to  zero.  This  is,  however,  a  misleading  example  and.  in  general, 
deterministic  optimal  control  is  far  from  being  a  trivial  by-product  of  stochastic  control,  which  is  in 
fact  in  some  respects  substantially  simpler.  Indeed,  much  of  it  concerns  the  "uniformly  elliptic"  case, 
where  the  "smoothing"  properties  of  Brownian  motion  make  dynamic  programming  in  its  simplest 
form  a  viable  technique  and  obviate  the  need  for  special  methods  to  handle  non-difTerent  iabilil  > .  In  this 
research  an  alternative  approach  is  developed,  in  which  stochastic  control  is  viewed  as  deterministic 
control  with  a  particular  form  of  constraint  structure.  Thus  the  distinction  between  "delerminisl ie" 
and  "stochastic"  is  in  principle  eradicated.  (In  practice,  of  course,  it  is  not,  since  the  form  of  t  In- 
constraints  leads  to  idiosyncratic  solution  techniques.)  The  first  to  espouse  this  point  of  view  were 
Kockafellar  and  Wets  [  1  (>].  and  <•  succession  of  papers  followed  in  the  stochastic  programming 
literature,  including  some  very  general  formulations  in  continuous  time.  The  term  s/m/nMo 
proyi  amnuiig  denotes  a  problem  in  which  the  cost  or  reward  is  given  as  some  explicit  function  of  a 
decision  process  arid  some  random  parameters.  I'ntil  recently  little  had  been  done  in  this  vein  in 
stochastic  control,  i.e.  problems  where  the-  cost  is  determined  implicitly  through  the  evolution  of  some- 
dynamical  system,  mainly  because  of  apparent  difTicunies  in  formulating  the  problem  correctly.  I  lu-se 
difficulties  have  now  been  overcome,  and  the  Kockafellar/Wets  approach  has  been  exnhued  with  some- 
thoroughness. 


Stochastic  optimization  problems  typically  take  the  form 


min 

(I  e  a> 


wliert'  u.  €  the  set  of  random  evnts.  2)  is  some  class  of  functions  d:il — F  (F  is  some  space  of 
divisions)  and  F  denotes  expectation  with  respect  to  a  probability  Pon  Q.  Note  that  this  problem  is 
truly  "stochastic"  only  when  isincompletely  known.  If  the  controller  knows  ^  in  advance  then 
(ignoring  technicalities)  ©  is  the  set  of  all  functions  d:Q — F  and  we  have 

min  EJ(d,u)  —  F  min 

d  €  ©  /  €  F 

rims  minimization  can  be  carried  out  separately  for  each  w  and  the  only  role  oft  he  probability  F  is  to 
average  the  result.  In  dynamic  optimization  F  is  typically  a  class  of  functions  such  asLx [T:f ’].  I 
being  the  "action  space"  and  T  a  time  set.  so  that  the  controls  are  stochastic  processes  1  In¬ 

most  basic  requirement  is  lliat  these  processes  respect  the  flow  of  information,  i.e.  depend  at  each  time 
only  on  what  has  been  observed  up  to  that  time.  In  fact,  this  is  a  linear  equality  ronshtunl.  In  see  l  hi- 
in  the  simplest  setting,  suppose  f)  =  {wj ,  *.>  —p  =  R.  T =  {0.  I }.  A  control  (/{/.-.  i  i-  then 

equivalent  to  an  8-vector  r  —  ( jc  j . r.  )  where 


and  the  cost  can  be  expressed  as 

‘I 

/•V(u.w)=  V  =-.g(j) 

i  —  I 

where  /e  =  P({w-}).  Let  .1  =  {-|.*.,|  and  suppose  that  at  time  0  we  have  no  observations  while  at 
tune  I  we  discover  whether  .4  or  .4f  has  occurred.  The  decision  rule  must  then  be  a  constant  at  time  U 
and  constant  on  .4  and  ,4r  at  lime  1.  But  this  means  that  r  must  satisfy  the  equality  constraint 


Hr  =  (I.  where 


1-10  0  0  0  0  0 
1  0-10  0  0  0  0 

H =  1  0  0-10  0  0  0 

0  0  0  0  1-10  0 
0  0  0  0  0  0  1-1 

Or.  pul  another  way.  j-  must  lie  in  a  certain  3-dimensional  subspace  S  of  R8.  There  is  therefore  a 
Lagrange  multiplier,  i.e.  a  5-vector  A  €  .V  ^  such  that  the  optimal  decision  is  a  global  minimum  of 
<j(t)  -r  A 1  // x .  The  multiplier  A  is  the  price  for  information  in  that  it  gives  the  incremental  decrease  in 
cost  that  is  available  ifthe  information  constraint  is  waived. 

In  this  research  these  ideas  are  extended  to  continuous-time  dynamical  systems,  i.e.  to  the  problem  of 
minimi/mg 

/••'[OO'V 

where  i  he  process  j  f  satisfies  the  controlled  stochastic  differential  equation 

di't  -  }(*/.  u()dt  f  <J\ * f)dii 

Here  ii  i  is  a  Brownian  motion  and  t if  is  the  tnon-anticipative)  control  process. 

In  tin1  mid  fid’s  Wong  and  Zakai  [30]  addressed  the  problems  of  approximating  stochastic  integral'  by 
ordinary  ones  and  approximating  the  solutions  of  stochastic  differential  equations  (SUL)  by  the 
solutions  of  sequences  of  ordinary  differential  equations  using  piecewise  linear  approximations  of  the 
Wiener  process.  Sussmann  ( 1 8j  continued  in  the  70’s  by  studying  when  can  the  solution  of  a  SDL  be 
defined  by  "extending  by  continuity  to  Cu“  noises  the  solution  of  a  C1  noise-driven  ordinary 
differential  equation.  1’sing  these  ideas.  Davis  [5]  reduced  the  LQG  problem  for  the  linear  controlled 
diffusions  driven  by  Wiener  processes  to  a  family  of  deterministic  linear  quadratic  optimal  control 
problems  (parametrized  by  the  paths  of  the  driving  Wiener  process)  with  nonanticipativity  equality 
constraint  on  the  set  of  admissible  possibly  anticipalive  controls.  First,  a  pathwise  optimal  cost  is 
deterministically  evaluated  for  each  of  the  problems  of  the  family  by  first  considering  driving  stochastic 
processes  with  ('*  paths  and  then  using  the  wt.rk  of  . Sussmann  [lt<]  to  extend  the  cost  to  ('"  paths  by 


ronliimily  .  Finally,  averaging  over  the  .sample  space  yields  the  usual  LQG  optima)  cost. 

The  report  will  approach  in  this  perspective  the  relation  between  nonlinear  deterministic  and  stochastic 
control  problems  by  reducing  the  nonlinear  stochastic  optimal  control  problem  to  a  family  of 
deterministic  optimal  control  problems  with  a  cost  containing  a  nonanticipativity  constraint.  Consider 
the  nonlinear  (nonanticipativc)  stochastic  optimal  control  problem 

dx,  =  f(x{.  u,)  dt  +  g(x, )  dwj  (0.1) 

x()  =  x(0) 

im  F  [6»(x(T'))j  :=  inf  F  J(u,w)  (0.2) 

it  €  -N  it  €  -N 

where  X  is  the  class  of  nonanticipativc  (adapted)  controls  with  values  in  a  compact  set  Tt  C  K1"  i-V 
will  be  defined  its  detail  in  the  next  section).  If  we  can  define  §(u,w)  for  non-adapted  (ami<  ipatixe) 
rout  rots  tt  €  ,4  with  values  in  the  compact  set  °U  C  Rm  (see  next  section  for  t  he  complete  dell  nit  ion  ol 
.X )  l  hen 

inf  I.  3ln.w)  =  F  inf  Jt  n  .,  w(-.w))  (()..!  t 

n  €  -A  u_  £  .4k  * 

provided  we  make  some  assumpl ion*  ensuring  that  the  infiiiium  on  the  right  is  attained  for  each  _  and 
the  function  assigning  to  each  ~  the  corresponding  minimizing  control  function  is  measurable.  .It  i-  1  In- 
class  of  mensurable  'll- valued  (deterministic)  controls.  This  requires  of  course  a  definition  lot  the 
solution  of  an  anticipative  SDK  (with  anticipative  drift)  which  will  be  done  in  Section  I  using  tin- 
decomposition  of  solutions  of  SDK's  (see  Kunita  [14.  p.  268]  and  Ocone  and  Pardonx  [15]).  (0.5)  can 
In-  used  to  solve  problem  (0.2)  by  solving  a  family  of  deterministic  optimal  control  problems  indexed 
by  £  Q  for  a  wider  class  of  controls  u  €  ^.  This  can  be  done  by  adding  to  the  cost  a  Lagrange 
multiplier  corresponding  to  the  nonanticipativity  constraint  as  a  linear  functional  of  the  controls  and 
by  .solving 


inf  M  [3('i^--w(-.w))  +  <AM.  u  .)>] 

ti  .  € 


for  each  «*•  €  Q  and  then  averaging  over  fi.  To  recover  (0.2)  we  would  like  the  above  minimum  lo  be 
attained  at  the  same  u(..  w)  at  which  (0.2)  is  minimized.  Assuming  (0.2)  has  an  optimal  feedback 
solution  u(t.  x,).  we  will  consider  the  nonantir  ipativity  constraint  as  an  integral  cost  term 

l 

A 1  ( t .  x(.  -•)  u_^.(t )  <lt 

U  , 

and  we  will  define  A  (t,  x.  -,)  (superscript  i  denotes  transpose)  so  that  A  (t.  xt.  .*•)  is  I. 


inlegrable  (i.e.  E 


I A  1  ( t .  x..  ^)|  dt 


0 


x  for  any  u(  £  A  where  xt  is  the  solution  of  (0.1) 


corresponding  to  u()  and  : 


i  i  j 


i 


arg  mf  [3(u  ••"(..  «.•)!  +  A 1  ( t .  x(t.  a.-),  ~)  u  .(t)dtj  = 
n  €  -'It  J 

*  U 

=  arg  inf  J(u.  w)  =  u*(t.  x(t.  -)) 
uEJf 

i:  a  (t.  x.  _)  =  it 


r  rr 

(lii)  A(t.  x.  *•)  is  -  adapted  (i.e.  future  adapted  :  is  the  sigma  algebra 

generated  by  the  future  increments  <  f  the  driving  Wiener  process  w(r.  >*;)  -  w(s. 
w):  t  <  t  <  s  <  I  ) 


(  ondit ions  (ii)  and  (iii)  imply 

i 

E  A 1  ( t,  x(t.  ut).  »•)  u(t,  w)  dt  -  0 

0 

for  mmanlicipative  controls  u(...)  £  Jf. 

Our  a|iproach  extends  the  Wong-'/akai  type  of  results  [20]  (which  show  how  solutions  of  stochastic 


differential  equations  arise  as  limits  of  solutions  of  sequences  of  ordinary  differential  equations)  from 

SDK  lo  optimal  control  problems  for  controlled  SDK  via  the  HJB  equation  of  dynamic  programming. 

In  the  case  when  nonanticipating  controls  appear  in  the  drift  the  Wong-Zakai  convergence  result  state's 

that  under  smoothness  and  boundedness  assumptions  on  the  coefficients  we  have  (  P-lim  denotes 

n— x 

limit  in  probability) 

xn(t.*')  =  x ( t .»') 


where 


dX  1  =  f(xn(t.o.).u(t.-))  +  g( x"(t .w))  ;  xn(0.^)=  x0  (0,1) 

cn(t.  «.•)  =  w(tk._-)  +  2"(t  -  tk)  (w(lk+].w)  -  w(tk.w)):  t  6  (lk.  tk+,)  (tl.o) 

tk  =  k/211:  k  =  U.  1  ....  [1/2"] 

dcn(t,w)  ,, 

dt  =  “  ('v(>k+1-')- w(tk..))  fort  e  [lk.tk+,] 


and 


dx(i.w)  =  f(x(t,^).u(t,*))  dt  +  |>f x( t .—>•))  odw(t.w)  ;  x(U.w)  =  xQ 
Tfx.u)  =  f(x.u)  -  i  gxg(x) 


where  “  o "  denotes  Stratonovich  differential  and  u(t,u,)  g  K  are  the  paths  of  the  nonanticipaiive 
control.  As  it  is  known  (O.fj)  can  be  written  in  Ito  form  as  (0.1)  by  adding  a  correction  term  to  the 
drift.  Let  us  consider  the  stochastic  optimal  control  problem  (0.1), (0.2).  The  dynamic  programming 
second  order  PDL  of  stochastic  optimal  control  [8,  p.  1 54]  gives  under  suitable  assumptions  the  value 
function  V(t,x)  E[0(x(T;l.x,u))]  (  x(T;t,x,^)  is  the  solution  of  (0.1)  at  time  T  started  at  time 

I  from  x)  : 


('•  x)  f(x.  .1)}  +  -‘tr  ggT)  =  0  :  V(T.  x)  =  ff(x) 


(0.7) 


This  is  shown  by  using  t hr  probabilistic  representation  formula  for  the  solution  of  (0.7).  Lei  us  consider 
now  the  sequence  of  pathwise  deterministic  optimal  control  problems  for  (0.4)  : 


inf  [0(xn(T.w»  + 

II  €  -Alj 


T 

(  An(t.xn(t.u,').w))Tu(i.ui)dt  ] 

0 


(O.s) 


where  A"  is  the  approximation  sequence  (corresponding  to  (0.5)  )  of  the  Lagrange  multiplier  process 
allowing  us  to  solve  (0.2)  over  (lie  enlarged  flats  of  possibly  anticipating  controls  by  solving  the  family 
of  pathwise  optimal  control  problems  o<  r  u  £  Jk  for  almost  all  w  £  Q 


inf 

ii  €  -4L 


T 

A '  (t.x(t.wt.-)u(t.u.)dt] 

0 


I  lie  value  function  VH(t.x.»|  :=  inf  (7?(xJ,(T.f..x.w))  + 

u  £  -4L 


T 

| (  An(r.x,,(r:l.x.-,).-'))1ii(r.u.)<lr  ] 
0 


of  (ll.i*)  satisfies  the  sequence  of  families  of  dynamic  programming  first  order  PDFs  (parametrized  by 


.*  /  ol  deterministic  optimal  control  : 


( t .  x .  u.' )  + 


min  { 
u  £  V 


()\l 

dx 


tt.  X .. 


.)  f(x.  u)  -+■  A(t,x,w)u} 


=  0 


V "(T.  x.w)  =  0( x ) 


(0.0) 


I  he  Lagrange  multiplier  process  that  will  be  introduced  in  the  report  will  give  an  answer  to  the 
intriguing  question  :  How  can  we  arrive  at  (0.7)  from  (0.9)  namely  what  is  the  "bridge"  Iretween  the 
second  order  PDK  of  stochastic  dynamic  programming  and  the  first  order  PI)K  of  deterministic 
dynamic  programming  ?  The  answer  is 

V(t.x)  =  K  P-lim  V  "( t .x.ui) 

II— x  ' 


II 


and  the  bridge  itself  which  fills  the  “gap”(in  the  terminology  of  [18])  between  stochastic  and 
deterministic  optimal  control  is  a  new  Hainilton-Jacobi- Bellman  backward  Stralonovich 


stochastic  partial  differential  equation  1  .at  will  be  introduced  and  studied  by  us 


d\'(t.  x..c) 


+  min  I 

u  e  'll 


dv 

i)\ 


(t.  x.*)  f  (x.  u)  +  A(t.x.w)u)  dt  +  —  ( t,  x.w)  g(x)  6  dw  |  =  I) 


\  (T,  x.-/)  =  0(x) 


(tl.lOt 


having  the  unique  global  solution  \"( I .x.w)=l>-jiii_i  Vn(t,x.w)  in  a  certain  space  of  Holder 
semimarl  ingales  to  be  defined  precisely  in  Section  1.  Our  deterministic  methods  will  concentrate  on  the 
robust  PDF  associated  to  (0.10)  defined  almost  surely 


0\\ 

<h 


(t.  + 


min  {  ( t 

u  €  Tl  *>>l 


(»/).u)  +  A(t. £.(//). _)||)=  0 


tit.  I  1  i 


W  (T.  ;>.-.■)  =  0  o  f7(/,) 


where  it  is  prosed  that  \\  { t . £ ( ( x ) _ 1  =  V(t.x.w)  .  We  use  the  almost  sure  decomposil iun  uf  the 

stochastic  flow  of  (0.1)  [I  I]  given  by 


X,(*l  =  S,  O  7/,(.\) 


di,(y)  =  g(£t(>'H  od"p  i0(yl  =  > 


(0.1-J) 


dr/,(x) 

"di- 


\T*J 


(. 


^’(Ojx))  f(f,  o  t;,(x).  u,)  :  t/0(x)  =  x(0) 


(0.1 1’") 


Ibis  decomposition  holds  on  some  Q'  C  0  with  for  which  £j(y)  is  a  global  flow  of 

dif feoiiiorphisins  as  we  will  see  later  -."i  The  robust  PDF.  (0.11)  is  the  dynamic  programming  equation 


for  the  pathwise  optimal  control  problems 


T 

inf  [0  o  i  j  t?;-,  )  +  A(t.f  (»/,).w)ii(t..u)di] 

li  6  .lb 

0 


subject  to  (0.12")  as  only  this  involves  controls. 

We  will  also  consider  the  anticipative  optimal  control  problem  (A=0)  which  will  be  solved  by  reduction 
to  pathwise  deterministic  control  via  the  stochastic  flow  decomposition  formula.  The  problem  in  this 
case  is  that  the  random  value  function  of  the  family  is  characterized  by  a  nonliner  backward  SIMM', 
which  does  not  always  have  a  global  stochastic  characteristics  solution.  We  give  conditions  ensuring 
this  by  turning  a  I.agrangian  submanifold  (see  the  stochastic  mechanics  of  Bismut  [it],  Arnold  [1])  into 
an  invariant  manifold  for  the  stochastic  haiiiiltonian  system  of  characteristic  equations.  I  lie  optimal 
control  i-  given  by  a  selection  lemma  and  it  formula  is  obtained  for  the  cost  of  information  on  the 
future  (i.e.  the  difference  between  the  uonattt icipat ive  and  anticipative  value  functions)  : 

A(t.x)  =  Y(t.x)  -  L'Y°(t.x.-) 


where  as  we  will  see  V  (t.X,*,) 


it  €  -'tl- 


0(xCl  :t.x..j))  satisfies  the  HJB  SPDK 


d\’**(t..x.»)  +  min  {^y— (t.x.^if (x.u)}di  +  (t.x.w)g(x)o  dw.  =  0 
n  £  Tl  ox  ox 


(0. 1:li 


Vu(T.x.w)  =  6(x) 


so  that  KV^(t.x.«/)  =  inf  E[0(x(T:t.x.*)]-  The  cost  of  perfect  information  known  in  the  stochastic 
tt€  A 

programming  literature  as  KYIM  (expected  value  of  perfect  information)  [16], [19]  is  a  measure  of  the 
effect  of  future  randomness  on  the  stochastic  optimal  control  problem. 

In  order  to  obtain  explicit  equations  and  formulae  for  the  cost  functions,  optimal  controls,  l.agrange 
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multipliers  and  for  the  cost  of  information  we  will  make  smoothness  and  boundedness  assumption* 
using  a  dynamic  programming  approach.  Most  of  these  assumptions  are  made  to  he  able  to  prove  the 
dynamic  programming  equation  for  the  almost  sure  optimal  control  by  which  the  anticipative  control 
problem  is  solved.  These  assumptions  ensure  this  equation  has  a  solution  (the  value  function)  which 
can  be  represented  in  terms  of  stochastic  flows.  The  same  applies  to  the  cost  of  perfect  information. 
Some  of  these  assumptions  ran  be  relaxed  in  particular  cases  as  we  will  see  in  sections  2.1  and  3.3  .  If 
we  give  up  these  computational  aims  we  can  weaken  the  assumptions  by  an  approach  based  on 
applying  pathwise  the  deterministic  maximum  principle  (see  Remark  4  after  Proposition  2.1  ). 

l  he  outline  of  the  report  is  as  follows.  In  Section  1  we  define  the  solution  of  a  stochastic  differential 
equation  with  anticipative  controls  in  the  drift  by  using  the  decomposition  of  t he  flow  of  a  SDK 
written  as  the  composition  of  the  stochastic  flow  of  the  diffusion  part  (which  does  not  involve  controls) 
with  the  flow  of  a  family  of  ordinary  differential  equations  (ODE)  parametrized  by  Controls  only 
appear  iu  this  random  ODE  and  they  ran  anticipate  as  stochastic  integrals  are  not  involved. In  out- 
paper  [lij  we  used  anticipative  stochastic  calculus  and  the  result  of  [12]  on  the  existence  and  uniqueness 
of  solutions  of  drift  anticipative  SDL's  in  a  suitable  Sobolev  space  over  the  Wiener  space.  This  approach 
require*  Wiener  smoothness  and  boundedness  assumptions  on  the  controls.  We  also  present  in  this 
Section  the  stochastic  characteristic*  method  which  allows  the  representation  of  solutions  of  SPUE*  in 
terms  of  stochastic  flows  of  SD1*.  T  his  representation  will  be  very  useful  in  finding  Poisson  bracket 
conditions  for  the  existence  of  g'oba1  solutions  and  for  proving  the  various  convergence  result*  for 
SPDKs  via  the  existing  convergence  results  for  SDEs.  In  Section  2  we  prove  the  Lagrange  multiplier 
theorem  giving  an  explicit  formula  lor  A(t.x.w)  iu  terms  of  the  unique  global  solution  of  a  linear 
backward  HJB  SPDE  showing  that  the  mul.iplier  has  all  the  properties  required  for  it  to  act  as  an 
equality  constraint  for  the  noiiauticipativity  constraint  for  control  processes.  We  consider  as  an 
example  the  uonanliripative  LQ(J  problem  which  is  solved  pathwise  by  determining  first  the  Lagrange 
multiplier  process.  In  Section  3  we  solve  the  anticipative  optimal  control  problem 


inf  E[(?(x(T)j 
u  €  A 


and  w  e  show  t  hat  the  value  function  of  the  path  wise  problems  by  means  of  which  it  is  solved  is  the 
llllk|lie  global  solution  of  a  nonlinear  backward  Stratonovich  SPDE.  We  impose  Poisson  bracket 
conditions  for  random  conservation  laws  or  for  the  invariance  of  the  Lagrangian  submanifold  for  the 
stochastic  characteristic  system  oi  the  .onlinear  SPDE.  Such  conditions  imply  the  existence  of  a  global 
characteristic  solution.  A  formula  is  obtained  for  the  cost  of  perfect  information  and  we  prove  that  t hi" 
is  zero  when  the  Lagrangian  submanifold  is  invariant  or  there  exists  a  time  varying  non  random 
conservation  law-  for  the  stochastic  characteristic  system  of  the  HJBSPDE  of  aiiliripative  optimal 
control.  We  consider  as  examples  the  anticipative  LQG  problem  and  a  scalar  nonlinear  anticipative 
optimal  control  problem.  We  show  that  there  exists  indeed  a  random  conservation  law  in  the  I.Q( •  case 
and  we  calculate  the  cost  of  perfect  information.  The  Lagrangian  submanifold  is  invariant  in  the 
nonlinear  problem  case  and  the  cost  of  perfect  information  is  zero. 


1  :  Anticipative  controls,  solutions  of  anticipative  SDE  and 
stochastic  characteristics  solutions  for  SPDE 


Let  T>1)  and  (Q,  TF.  (*5 ( )y  <  ,  ^  -j-.  P.  (w()y  <  (  <  j)  denote  the  canonical  d-fold  Wiener  space,  i.e.  U 
=  ('([0,  T].  R^).  wt(-;)  =  -;(t)  is  the  coordinate  process,  (TF^)  is  the  natural  filtration  of  (w  ).  P  is 
Wiener  measure,  TF  is  the  P-completion  of  TF?j  and.  for  each  t  €  [0,  T],  TFf  is  TF^1  completed  with  all  P- 
liull  sets  of  “7.  Thus  (w. )  is  a  standard  d-dimensiona!  Brownian  motion.  Define  Q=[0.T]xf}.  “3F = “tBfO. 
T]  x  7,  P  =  LebxP  and  let  TP  be  the  rr-field  of  TF,-predictable  sets  in  Q.  Fix  an  integer  m  and  define  A 
to  be  the  set  of  functions  u  :  [0,  T]  x  Q — TL  C  Rm  which  are  measurable  with  respect  to  the  product  it- 
field  TB[l).  T]xTF  =  TF  where  Tl  is  compact.  Define  also  K  as  the  set  of  functions  u:  [0.  1]  x  0  — 
TL  C  Rm  which  are  measurable  with  respect  to  TP  and  thus  TF,  adapted.  Consider  the  nonlinear 
stoehast ic  system 

d  x,  =7(x(.  u,  )dt  +  g(x,)  o  dw,,  xu  =  x  £  R1*  (1.1) 

f(.x.  u)  =  fix.  u)-i^  g.  6.(x)  ;g.  = 

i=l  x 

for  a  nonanticipative  control  n,  €  -N"  the  solution  of  (1.1)  is  defined  via  Stratouov i«Ti  stochastic 
integral  equal ion 


f(.xT.  ur)dr  + 


g(.xr)  o  dw, 


For  anticipative  controls  ur  £  -X  we  in'.roduce  the  following  definition  assuming  for  the  moment  that  g 
i>  Cj-,  an<*  bounded:  f  is  bounded  a  id  '  in  (x.  u). 


Definition  1 


1'or  ut  €-4  we  call  a  solution  of  (1.1)  the  process  x,  defined  almost  surely  (i.e.  for  -  6  U1  C.  it  . 
P(Q')=1  )  by 


x,(x)  =  °  »?t(x) 


(I.'.') 


where 


d?t(y)  =  g(Ct(y))  0  dw(,  £q(v  )  =  v 


(].:!) 


d'?t(x) 

dt 


(11) 


'/()(  X  )  =  X 


1  lie  solution  ( 1.1')  is  defined  almost  surely  because  £((y)  is  flow  of  diffeomorphisnts  a.s.  so  i/,  is  defined 
almost  surely  too. We  see  that  this  solution  is  well  defined  whether  or  not  u,  is  adapted  because  ?/(  is 
not  defiled  by  stochastic  integrals  and  f,  does  not  depend  on  the  control.  This  definition  implies  that 
we  also  defined,  in  a  particular  case,  a  generalized  Stratonovich  integral 

i 

J  g(xr)  •  dwr 
0 

for  xr  the  solution  of  (’2)  to  be 


j  g(xr)  •  dwr  :=  xt  -  x() 

0 


j  f(xT.  uT)  dr 
0 


1  his  •‘decomposition  of  the  solution"  of  (1.1)  -  approach  is  used  in  [15]  for  defining  solutions  to  SDK 
with  anticipative  drift  .  but  a  different  generalized  Stratonovich  integral  is  considered  there  which 
requires  additional  smoothness  assumptions.  I'sing  the  anticipative  Ito  formula,  (1.2)  can  )«■  shown  to 
l>e  the  solution  of  (1.1)  in  the  sense  o:'  \)cono  and  Pardoux  if  additional  smoothness  is  im|H»cd  on  f.g 


and  t he  Anticipative  controls  have  bounded  Wiener  space  derivative  |Dsu(t.*;)|  <  M.  We  pursue<l  ilii> 
anticipative  stochastic  calculus  approach  in  [6].  We  assume  g  is  so  lhal  the  solution  of  I  !.:()  exists 
for  =  y  and  £,(v)  is  a  (d  flow  of  uitfeomorphisms  a.s.  ;thus  is  well  defined  for  all  (t.  y) 

a.s.  according  to  Hismut  [.'i.  p.'tOj.  I'nder  these  assumptions  ,  (1.4)  has  a  global  solution  (see  Ocoue 
atxl  Fardoux  [15]). 


We  present  next  the  notion  of  stochastic  characteristics  (global)  solution  (developed  in  Kunita  [12.111]) 
which  will  be  used  throughout  for  the  stochastic  partial  differential  equations  involved  in  our  approach. 
Consider  the  first  order  Stratonovich  nonlinear  SPDE  for  x  e  Rd,  t  €  R+  with  initial  condition 


dv(t.x)  =  F(l.x.^  )dt 
v(t).x)  =  fl(x) 


>1 


y.  <i:(l  x 
j=l  J 


•|)odw| 


(1.5) 


with  F(t.x.p)  continuous  in  (t.x.p)  and  a  fn‘+I,Q  -  function  of  (x.p)  (i.e.  m+I-times  continuously 

t  h 

differentiable  in  (x.p)  with  r>- Holder  continuous  ni+l,n  partial  derivatives,  a  >  0).  (ij(t.x..p)  j  =  I . <1 

continuous  in  (t.x.p).  continuously  differentiable  in  t  and  +  —functions  of  (x.p)  for  m  >  :i  and 
0  is  continuously  differentiable.  «((-.)  is  a  standard  d-dimensional  Brownian  motion  as  above. 


A  random  field  v(t.x)  defined  for  all  t  £  R+:  x  €  R-  will  be  said  global  C"1'  —  prouss  if  for  almost  all 


€  $?  v(l. •..»•)  is  a  ( 'n -  function  for  all  I  G  R^.  with  continuous  in  (t.x)  partial 
l.,k|  <  (II  where  =  (£-)  '  ...  k  \  =  k,  +  ...  kj.A  global 


derivatives 


tlx*' 


(,m‘°  -  process  is  a  global  .  senwnartingaU  if  — — -  f  |  k  |  <  m  are  semimartingales  for  each 

dtk 

x£  Rd- 

Definition  2  A  random  field  v(t,x)  is  a  global  C^'a  solution  of  (1.5)  if  it  is  a  global 
( 'l'°  —  semimartingale  and  it  satisfies  for  all  (t.x)  £  R^.  x  R^ 


t  j  t 

v(t.x)=  0(x)  +  J F^r.x.  +  2l  |Gj(r’x' ^(T’x))odwJr  as- 


j=>  6 


18 


Ii  is  proved  in  [12.13]  that  the  linear  SPDK  with  initial  condition  0{  • )  of  class  2  <  I  <  in 

d 

f  dv(t.x)  =  F(i.x)t^(I.x)  dt  +  (.ij(t,x)^(t,x)odw| 

1  j=‘ 

l  v(O.x)  =  0(x)  (l.(i| 

having  F  continuous  in  (t,x)  and  of  class  an(j  Gj  continuous  in  (t,x),  of  class  (m  >  ;() 

has  a  global  (unique)  solution  with  .1  <  a.  Generally  (1.5)  has  only  a  local  solution  v(t.x)  up 

in  a  stopping  time  t  <  T(x)  which  requires  the  definition  of  local  random  fields  and  local 
C1".'1  -  semi  martingales  [12.13].  This  solution  is  represented  as 


V(I.X)  =  Uy  O  Oj’(t)  (1.7) 

where  rt(x)  =  ( \.f>x(x )).  \,(xl  =  \,(.\.(lx(x)).  t x^x)  =  i>((x.0(x).0x(x))  are  the  flows  tin- 

s/nr  hnstn  >  rhnracUn>hr\  w/.s/r  in  with  general  initial  condition 


dr,  =  ~  Kp(t.^,.\,)dt  +  Yi 
j=l 


(  I  ■ 

.IP 


(t.^,.\t)  °dw{ 


d 

<i\,  =  fx(i.;-,.\,)«ii  +  Y  Gj  °dwJ, 


(  I.K) 


tl.il) 


di>,  =  (F(l.£,.\,)  -  F p( t .v~t . \ t )  \t>d’  +  51  (Gj(t,^,.\t)  -G:  (lv,.\,)\,)°dw{ 

j=l 


(l.l«) 


V>0  =  a.  x0  =  l»  .  £-„  =  c 

Here  Fp(t.x.p)  denotes  the  vector  of  partial  derivatives  with  respect  to  p.  The  reason  for  the  solution 
(1.7)  being  only  locally  defined  up  to  a  stopping  time  t  6T(x)  is  that  Vj(x)  's  in  general  only  a  local 


How  of  diffeomorphisms  because  of  the  "coupling"  between  (1.8)  —  (1.9). 


We  will  be  interested  only  in  global  solutions  as  these  will  characterize  value  functions  of  various 
optimal  control  problems  .  That  is  why  we  do  not  present  in  this  introduction  the  local  theory 
developed  in  [l'J.l.'lj.  We  will  be  forced  to  fino  conditions  ensuring  (1.8)  —  (1.10)  lias  certain  (almost 
surely)  invariant  submanifolds  "decoupling"  from  one  another  the  stochastic  characteristics  and 
ensuring  the  existence  of  ,?|*(x)  for  all  t  €  a.s.  thus  turning  (1.7)  into  a  global  solution.  From  this 
point  of  view  the  Poisson  bracket  conditions  of  §3.1  represent  new  results  for  the  existence  of  global 
solutions  for  nonlinear  SPDE's.  'lie  coefficients  of  our  SPDE's  will  be  assumed  to  have  bounded 
derivatives  (thus  o=l)  and  all  our  solutions  w  ill  be  global  C" for  3  <  1.  We  will  term  them  I >ilohnli 
C~  Mtlutuni*  omitting  3.  Also,  we  will  use  backward  SPDF.'s.  Using  Ito’s  backward  formula  [I  I]  and 
replacing  in  (1.8)  -  (1.10)  the  forward  Stratonovich  integral  by  the  backward  Stratonovich  integral, 
the  characteristics  representat ion  of  solution  of  a  backward  SPDK  is  again  (1.7)  but  with  backward 


characteristics. 


2  :  Families  of  deterministic  control  problems 


and  nonanticipativity  constraints  via  Lagrange  multipliers 


We  shall  show  that  the  standard  tnonan'icipative)  stochastic  optimal  control  problem 


inf  E[*(x(T))] 

n  €  Jf 


for  (1.1)  can  Ire  solved  by  allowing  a  larger  class  of  controls  u  €  -4  (possibly  anticipative).  introducing 
nonanticipativity  constraints  in  the  cost  via  Lagrange  multipliers  and  solving  instead  a  family  of 
deterministic  optimal  control  problems  parametrized  by  the  sample  functions  of  the  Wiener  process  and 
of  the  Lagrange  multipliers.  A(t.  x.  -.)  with  A  :[U.  T]  x  R^  x  Si— R1"  : 


r 

inf  [0(xT(.\.  *•)) -i-  A'  ( 
tt  €  -4L 


7.  x.tx.  .*•).  *)  u(r)  dr] 


with  x((x.  «*•),  the  solution  of  (1.1)  for  anticipative  controls  as  defined  in  Definition  1.  x,(x.  -I  =  c 
t/,(x).  As  only  r/,(x)  depends  on  u,  we  get  the  equivalent  family  of  problems  for  £  U'  with  U'  defined 
in  Definition  1 


inf  [0  o  (i/  ..)  +  Ar  | r.  //,.  w)  u(r)  dr] 
u  G  -■tb  J 

0 


=  (^f)“I(r,t)  f(^t  0  ^t-  ut>-  '?o  =  x 


where  we  dropped  x  from  the  notation  »;((x)  for  simplicity  and 


A  (r,  r;,  *;)  :=  A(r,  £r(r;).  wl  (--"») 

By  averaging  (2.3)  over  the  sample  space  for  suitably  chosen  L*  Lagrange  multipliers  one  would  then 
like  to  obtain  the  optimal  cost  (2.1).  We  will  state  and  prove  in  this  section  a  Lagrange  multiplier 


theorem  showing  how  to  define  A(r.  ip  so  that  it  will  act  as  a  nonanticipativity  const raint  in  a 
family  of  deterministic  (anticipativo)  optimal  control  problems. 


We  first  make  trie  following  assn  nipt  ions 


(a)  f  is  hounded  in  x  and  u  and  with  continuous  and  bounded  in  x  and  u  mixed  partial 
derivatives  up  to  order  5  (i.e.  Cj*):  g  is  bounded  and  C^;  6  is  . 


(l>)  For  every  y  €  R  .  ^  (gg'(x))  -  y-v-  >  0  for  all  x  £  R  . 
t.  j=l 


(cj  The  (nonanticipative)  stochastic  optimal  control  problem 


clx,  =  f(xt.  II, )  (It  4  g(\,  I  dw 

I’)  x()  =  x 

inf  „  F.[D(x(T))i 
u  e  x 


lias  a  feedback  solution  u*(t.  x)  which  is  continuous  and  C1  in  t.  (  ^  in  x. 
ti*(t.  x)  £  int  Hi  for  all  (t.  x)  €  [O.'l]  x  where  'll  is  the  control  value  set 
assumed  to  be  a  compact  subset  of  R,n. 


(d)  I  lie  matrix  with  (i.j)  element  t’iveii  in  repeated  indices  summation  convention  notation 


d(0o£T  o  ti,  (r?)]  /  /d£,\  .  \  d2fi 

hy  - 3^- - (  (-^)  ■  (»/)X,  <  W’  u)  has  characteristic  values 

bounded  below  by  some  -)>Q  for  all  (t  r).  u)  £  [0,  T]  x  R^  x  Hi  a.s.  where  Cj*  is  the 
inverse  flow  of 


=  brJ (t  t("»  t ('/)).  n  <t.  €t(vt(ri))D  ]T  gixgj( 


(2-7) 


(€,(«■, (>?)))  -  U  (*tU\(v)).  u*(i.  ^t(t  t(7?))))  u*(t.  ft(^'t( ’?)))]:  *r W  =  '> 


Assumption  ( cl )  .as  will  be  seen  in  the  next  section  ensures  the  strict  convexity  in  n  of 
^r^-( t t/)f(f,( >/). n )  which  together  with  the  particular  choice  of  the  Lagrange  multiplier  yield' 
u*(t.x)  as  the  unique  minitnizer  of 


{w(t’,,)(^r)1(',)fUf«{',Ku,+r(t-^)u} 


The  characterist ic  representation  is  used  to  write  the  strict  convexity  condition  in  the  form  appearing 
in  (tl). 


Assumptions  (a.  b.  c)  imply  that  the  value  function  of  the  problem  (2.6).  Y*(t.  x).  is  the  C  •  -  (j.e 
C*  in  t.  (  ■  in  x)  uni(|iie  solution  of 


ijrzr1  +  ^«u  C (t- x)  +  itr  <$f  =  « 


Y*|  I  .  X)  =  0{\) 


The  assumption  that  f  and  g  must  be  bounded  can  be  relaxed  only  as  far  as  allowing  them  to  ha\e 
"stiblinear  growth"  [ I •’>]  i.e.  V.  <  >0  3.  K,  s.t.  |  g(.\)  |  <  k(  (1+  |  x  I  his  is  because  ol  using  the 

decompo-ii ion  x,  =  £,  o  t;(  and  because  of  needing  to  ensure  that  tq  (given  by  (2.1))  does  not  explode. 


2.1  Main  results 


Our  main  Lagrange  multiplier  theorem  is  the  following 

Theorem  2.1 

Consider  the  following  family  of  deterministic  optimal  control  problems  indexed  by  *  6  V.'  C  it 
P(Q')=I  (  see  Definition  1) 


'll  WV- 


(2.8) 


( !»'*') 


T 


inf  [ 
ii  6  JIj 


0 


A‘(t.  i/r  *)  u ( t  )dt  +  0  o  £t(//  j.)] 


where  £.(>/)  is  the  solution  of 


‘•st  =  >  0  ‘l"'i  ( 

Ml  =  'I 


Assume  (it)  -  (d)  and  define 


A‘(t.  ’/.  -)  :=  ■ 


<>{<>  °  C-i 


0  (. 


(hi 


—  r„  (i :,(»/)•  u*(t.  {,{>;))) 


(2.10) 


C>f; 


where  fu  =  1  <  i  <  d.  1  <  j  <  lit. 


Then  u*(t.  £,(?/))  is  optimal  for  (Pw)  a.s.  and  so  u*(t.  x)  is  optimal  for  (2.2)  and  if  we  denote  by  \V(t. 
>/)  the  value  function  of  (I’**)  we  have 
VV(l.  ij)  =  0  of,  o  t' 1  ( »/ ) 


2-4 


r 


EW(t.  ^(x))  =  V*(t .  x) 

nT(t.{i’(x))  =  o  cj.ii) 

and  Y(t.  x)  :=  VY(t.  £j*(x))  is  the  value  function  for  1  lie  problem  (1.2)-(  1.3)-(2.2). 

The  basic  idea  in  the  proof  of  this  theorem  .s  to  use  first  the  deterministic  dynamic  programming 
approach  pathwise  for  (2.3). (2. -1)  to  obtain  a  random  (robust)  HJB  PDE.  The  Lagrange  multiplier  will 
be  chosen  so  that  the  minimizer  in  the  HJB  PDE  is  the  same  as  the  nonanticipative  (feedback)  optimal 
control  u*(t.£.(ij))  assumed  to  take  values  in  the  interior  of  TL.  For  this  we  need  the  strict  com  exit  y  in 
it  of  the  expression  to  be  minimized  in  HJB  PDE  and  A  to  be  such  that  the  derivative  in  u  of  tld> 
expression  vanishes  at  u*(t. £.(»/))  (interior  minimum).  The  characteristics  method  will  be  used  to  i  nsure 
that  there  exists  a  c'"  solution  of  this  pathwise  HJB  PDE  so  that  using  the  verification  theorem 
[8.p.87]  u*(t.it(77))  will  turn  out  to  be  optimal  for  (2.2).To  characterize  the  value  function  \Y(t.;/)  in 
terms  of  x  as  solution  of  a  HJB  S1M)E  we  will  approximate  the  Wiener  process  by  linear  interpolation 
and  we  will  show  that  HJB  SPDE  appears  as  limit  of  a  sequence  of  random  HJB  PDE's. Averaging 
HJB  SPDE  we  gel  the  second  order  parabolic  PDE  of  stochastic  dynamic  programming.  This  way  our 
proof  extends  the  Wong-Zakai  type  remits  [20]  from  differential  equations  to  optimal  control  problems 
fdynamie  programming  equations)! see  also  the  remark  after  the  proof  of  Theorem  2.1  and  [."»]  where  the 
extension  by  continuity  of  the  cost  to  C1*  paths  was  used).  It  is  for  this  very  important  reason  that  we 
will  prefer  a  rather  lengthy  approximation  argument  for  the  proof  of  Proposition  2.1  below.  One  can 
try  to  obtain  the  value  function  in  terms  of  x  by  proving  a  generalization  of  Ito  extended  rule  to 
calculate  the  differential  of  W( t ' ( x ))  as  the  random  Held  W(t,t/)  is  non  adapted.  This  will  directly 
show  that  the  random  HJB  PDE  is  the  robust  equation  associated  to  HJB  SPDE  via  the 
transformation  x={,(t/)  given  by  the  solution  decomposition  formula  (1.2).  We  used  the  ant  ■''•native 
Ito  rule  of  Ocone  and  Pardoux  [15]  which  requires  Wiener  derivative  growth  estimates  for  the  random 
field  W(t.»>).  This  approach  was  developed  in  [6]. 

We  start  by  proving  the  following  result  which  is  essential  in  the  proof  of  Theorem  2.1 
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Proposition  2.1 


Assume  (a)-(d).  The  value  function  of  the  problems  (P1^)  with  A  (t,  rj,  ^;)  defined  by  (2.10)  is  the 
unique  0 ' solution  of  the  random  PDE 

In  (t'  n)  +  (t< T,)  (S’1(n)  ff  u'(r,)'  u*(t'  ^(T,))) 

d 

-  7  E  gix  Si^t  (?/)J  -  (t ;,(>/).  u*(t.  ^t(7;))  „*(t.  £t(ij)))]  =  0  (2.12) 

i=  J 

W  (T,  7/)  =  6  o  Tr('/) 

where  ^ )  is  the  unique  nonexploding  solution  of  (2.9).  V(t,  x)  :=  W(l,  fj*  (x))  is  the  unique  C“ 
solution  of  the  backward  stochastic  partial  differential  equation  (SPDE) 


d\'(  t .  x)  +  ^  (t.  x)  [f(  x .  u*(t.  x )  i  -  ^  (x.  u*(t.  x))  tt*(  t .  x) 
c)\  (hi 

d 

■  i  E  Si  .  Sj(x)]  dt  +  ^  (t.  Xj  gtxjodw,  =  0 
i  =  1 


VI  T.  x)  =  0{ x ) 


(2.13) 


or  m  iio  backward  form 


d\  (t.  x)  +  (i.  x)  [f( x.  u*(t.  xj)  -  (x.  u*(t.  x);  u*(t,  x)  ] 

+  \  ,r  (^4  (l'  x)  W&'w)*  <h  +  ^  (I'  x)  g(x>  dwt  =  0 

V(T.  x)  =  0(x)  (2.11) 


Remarka 

1.  (2.12)  is  thus  the  "robust  equation"  associated  with  the  SPDE  (2.13)  (see  Cannarsa  and 

Vespri  [-1]) 


"d“  and  "  6  cl"  denote  t lie  differentials  eorresponding  respectively  to  Ito's  and 
Stratonovich's  backward  integrals  (see  Kunita  [14]) 

A  stochastic  Hamilton-Jacobi  partial  differential  equation  was  considered  in  Bismtit 
[3.  |>.  323]  in  the  context  of  ranch.  >  mechanics 

A  different  approaclt  can  be  pursued  based  on  pathwise  application  of  the  deterministic 
maximum  principle.  The  Lagrange  multiplier  can  then  be  defined  in  terms  of  a  random 
adjoint  process.  As  such  an  approach  will  not  involve  SPDE’s  and  thus  we  will  not  make 
use  of  stochastic  characteristics  the  smoothness  assumptions  can  be  considerably 


2.2  Proof  of  the  main  results 


Two  lemmas  are  required  in  order  lo  prove  Pioposition  2.1  and  thus  Theorem  2.1. 
Lemma  2.1 

Consider  the  sequence  of  random  PDE's 


AW11  .  /)u/n  /££? 

cit 


(t.  r?)  +  (t,  rj)  [f  (£"(»?),  U*(t,  ££(»?)) 


1  L  8ix  gi^l1  ('?))  -  £  <>*('.  sf!‘(»?)))  u*(t,  ^(1?))]  =  0 


\V"(T.  11)  =  0  O  {!}.(,,) 


(2. 15) 


with 


Tff  ('/)  =  g  ({{'(»/)  t  " 

s|j  ('/)  =  ’/  (2.  Hi) 

where  i"(t.  *■)  is  the  piecewise  linear  approximation  ("linear  interpolation")  of  the  d-dimensional 
W  iener  process  w(t, 

'  "(I-  *)  =  "(lk)  +  '  *■);>  (»'(«k+])  *  W(lk)):  1  €  flk’  lk+l> 

where  tj.  =  k/2n;  k  =  0.  1  ....  [T/2n]  so  that 

r,,(t.  w)  =  2n  (w(tk+1)  -  w(tj.))  (2.17) 

for  i  6  l«k.  tk+I]  . 

f  tuler  the  assumptions  (a.  r)  the  sequences  of  random  PDE's  (2.15)  have  unique  C^‘~  solution  for  each 


2ft 


M  €  N  and  Vn(t.  x)  :=  Wn(t.  (fJY'lx))  satisfies  : 


tfV11 


(t’ x)  +  (1,  x)  (x'  u*(,’x))  -  5  E  gi .  gi(x)  ■  frr (x'  u*(i>x^ 


<9f 


i=l 


X 


ll*(t.  X )]  + 


dvn 

dx 


g(x)  V ■“  - 


0 


Vn(T.  x)  =  0(x)  . 


Proof 

For  the  'piecewise'  linear  approximations  v”  considered  above  we  have 


av2 (t  x) _ m  (u{^y  (x)) ,  ml (t  (<nri(x)) 

(h  ll,X)_  d\  Or,  (X,)  dt 


=  -  (*•  («" )''(*))  (f(x-  "’i1-  x»  -  3  E  gix  gi(x> 

jjj;  (x.  »*(i.  x))  i,*(t.  x)]  -  (,.  (f'T^x))  (^y1  ((^r^x))  g(x)  <  " 


because 


H(i['r‘(x) 


<it  \  dx 


^1'  ’(xj)  g(x) 


I  sing  now 


^  C.  X)  =  ¥(..(«!')•' 


(>x 


V  V 


('  -§x'  (x))1  (§■)!  ( ( ^ *t’ )' 1  ( x ) )  =  In  (due  to  (a),  (x)  is  a.s.  a  flow  of 

diffeomorphisms  and  {J1  o  (£(n)'^(-v)=x  ) 


■J9 


wt>  get  the  required  sequence  of  random  PDE's  for  Vu(t.  x).  To  see  that  indeed  under  the  assumption: 


made.  \Vn(t,  r))  and  V"(t,  x)  exist  and  are  a.s.  we  consider  the  corresponding  characteristic 

equations  (obtained  by  replacing  ^(t in  (2.7)  by  to  get  rj '(//))  and 

representations  of  the  solutions  for  (2.i5)  and  (2.18).  Reasoning  similarly  to  (14.  Lemma  ti.2(-1)  ]  for 
the  integral  form  of  random  ordinary  differential  equations  we  write  forward  equations  for  the  inverse 
of  the  flow  of  characteristics  for  each  fixed  t  €  [0.  T]): 

ufr’M  =  ;  vfoui*  )'*(./)  =  n 

where 


tMt yj\ri)  fd$ 


=  «t [fi^ut!',)-^?)).  u*(s.  z"  ((v^r1^)))) 


■  7  Z  *ix  ~  ((?  1(1  Is)'1^))-  u*(s<  <S  ((t'l’j'V/)))) 


u’(s.  4'(u!'srl  <//)»]  :=  a"(s.  (t';’srl(r/). 


(t  J’j  r1  in)  =  n ;  1  < s  <  t 


and  respectively 

(cj'r’u)  =(<,-;,Tri(x):<;io(c!l;',(x)-x 

where 

d-\l  ~  =  KUu)'^*)-  uTs)'^)')  -ils;  gi((0'!(*)) 

i=i  x 

-  g  ((c['sr,(x).  u*(s.  (<;"  )-'(x»  ..•(*.  (^r*(x))  +  g((crs)'1(x))  vS 

(Cj't  r*(x)  =  x  :  t  <s<T  (2.20) 

For  each  t.  n.  for  almost  all  w  g  fi  (2.15)  and  (2.20)  have  unique  non-exploding  solutions  on  [(.  T] 
under  (a.  c).  It  is  clear  that  for  bounded  f  which  is  in  (x.  u)  and  bounded  g  which  is  ( 'j\  (2.20)  has 
for  every  n  g  N.  for  almost  all  g  Q  and  x  g  a  unique  solution  on  [t.  TJ  (3.  p.  .’{8].  To  see  that 


(2.19)  has  a  non-exploding  solution  on  [t.  T]  we  have  to  show  that  an(s,  x,  u)  satisfies  a  linear  growth 
condition  for  each  n  a.s.  YVe  know  from  [3.  p.39]  and  [15,  p.  57]  that  for  every  t.T.  J  >  0 


I  (S'1!*)  I  <  «"(*•)  d  +  I  X  |V  :  S  e  [t.  T] 


(2.21) 


for  some  family  of  random  variables  |f"(s)}  n  6  M  which  is  uniformly  bounded  in  L j,  for  all  p  >  I  i.e. 


sup  E  |  |p  <  oc  (2.22) 

n  €  N 


and  such  that  for  every  .?>()  and  o  >  0  there  exist  constants  C  ^  for  which 


P(C“  >  f) 


< 


C».J 

r 


(2.23) 


for  any  C>0.  in  fact  in  [15]  it  is  shown  that  (-r^Y*(x)  satisfies  (2.21)  using  E  sup  |  £  (xl  |p  <  (',.(1 

S<T  1 

+  j  x  jJ)p/‘  for  p  >  2  and  Sobolev's  inequality.  I’sing  the  fact  that 


£s‘(x)  -  X  =  l„(s.  «,)  where  E  [sup  |  I.  (s.  ^)  |p]  <  x 

s  <  l 


from  Ikeda  and  YVatanabc  [10,  p.  500]  one  shows  that  also  E  sup  |  £?(x)  |q 

s  <  T 

<  ('Jj(  I  -t  I  x|-’)q/,_  for  <i  >  2.  Here  c,j.  c[j  are  constants  for  each  fixed  <|  >  2.  n  6  \. 


Hue  to  the  assumptions  we  made  and  due  to  (2.21)  in  which  we  take  i 

d 

|  F"(s.  x.  ~)  |  :=  |  f(^'(x),  u*(s.  f“(x)))  -  \  £  gj 

•tr  i  =  l  X 

■|(Ox).u'(s,  cj(x)))u*(s,  *?(x))|  <  M 


(2.21) 


and 


a"(s.  x.  -)!<|  ($^'<x)  |  |  En(s.  x.  -»)  |  <  £"(->)  (1  +  |  x 


a.s. 


(2.25) 
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for  a  uniformly  Lp  bounded  family  of  random  variables  {£n(u>)}  n  €  N  for  any  p  >  I.  The  a.s.  linear 
growth  of  an(s,  x,  w)  (2.25)  ensures  that  the  solution  of  (2.19)  does  not  explode. 


Proof 

For  f  and  g  satisfying  (a.  ( )  we  get  fot  each  t  <  T 

k‘tT  )‘‘U)  -  CtT<x))  (2.HI ) 

where  t.J*(x)  satisfies 
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(2-32) 


<is  <;;*(*)  =  [f(c;'u),  u*  (S.  t-;](x»)  -  i  53  gj  gj(c^(x)) 

i=i  x 

-  §ft  (CM  u*<s-  Cli(x»)  «*(s.  «.ts(x)))  ds  +  g(Ctl(x))  o  dws 
c;j(x)  =  x:  t  <  s  <  T 

This  follows  by  applying  the  convergence  result  for  the  inverse  of  flows  of  SDKs  with  Ixnmdrd 
coefficients  [3,  p.66  and  p.39]. 

To  prove  ( 2.28)  we  use  now  the  representation  (2.27)  of  the  solution  of  (2.18)  given  by  the 
characteristics  method.  Assumptions  (a.  c)  ensure  the  existence  and  uniqueness  of  V"(t.  x)  for  each  n 
€  N  and  due  to  (2.31)  and  (2.27)  we  have  for  each  t  g  [0.  T] 


V"(t.  x)  —  Ho  Cj'f (x )  (2.33) 

uniformly  in  x  on  compact  subsets  of  Rd.  But  under  the  assumptions  made  we  know  from  Kunita  [12. 
Section  VI]  that  the  unique  ( solution  of  (11)  is  V(t,  x)  =  d  o  ([[*  (x)  =  0  o  C|-j-( x)  where  C,  ( x ) 
srt!  isfit** 


d  k',(x>  =  W(,(x).  «•  (t.  <,(*)))  -  i  8ix8i(Ct(x))  -  (Ct(x), 

u*(t.  <,(X)))  C|(x))]  dt  +  g(C,(x))  O  dw,  (2.31. 

C  j-(x)  =  x  . 

and  C;‘(x)  is  the  unique  solution  of  (2.32)  for  s  =  T  (Kunita  [14])  so  that  (2.28)  follows.  The  flow  of 
(2.34)  was  denoted  k'j/pfx)  as  it  has  terminal  condition  at  T.To  prove  (2.29).  (2.30)  we  use  the 
representation  (2.26)  given  again  by  the  characteristics  method  for  PDE.  We  will  prove  first  that  for 


each  I  £  [0,  T] 


(2.35) 


uniformly  in  i/  on  compact  subsets  of  R^.  (2.35)  will  then  be  used  in  proving  that  the  following 

convergences  are  valid  uniformly  on  compact  sets  of  R^,  for  each  t  €  [0,  T): 


VVn  (t.  (O' ’(*))  =  0°^o  (*-“Tr1  O  (^r’ix)  2.  W(t,  Q*(x))  =  «o£to  v;ij.  o  £y*(x) 

(2.3ti) 

and 

VV"  (t.  it)  =  0  o  {!J-  o  (t'”T)'1  ('/)  —  W(t,  T))  =  6  o  o  V’^y(r>)  (2.3<i') 

where 


d  fysiX)  =  -  g(£y|,tX))  O  dw' 
‘\)\)\  X )  =  x  :  0  <  -  <  t 

=  wt  -  wt-> 

«i‘  (X)  =  Zq\  (X) 


^4r'(x) 

ds 


s-8i(01w)d-!e 


t.ll 


‘'Os 


ds 


({[jor‘(x)  =  x  :0<s<t 

t.n  _  n  n 
'  s  =  ‘  t  '  *’U 

(«!’)' 1  (x)  =  (O)'1  <x) 


(2.371 


(2.38) 


and 


!iidT^  =  (§)■'  Ut*(^  lf(«s(trt!l(0)).  t.*(s,  «s(^(o)))) 
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35 


sup 

'  <  r  <  / 

i)  6  K 


_k,n 

rH 


lIlr^rWKr)'1 


(V)  +  (1  -  *?TM>  ( v'iT )  '  <'/)•  -■) 


x  |(c'P  )**  ('/)  -  </  7 i  ('/)  i  <1  t  <  'I’stip  |  an(r,  x,  at)  -  a  (r.  x,  w)  | 

t  <  T  <  T 

I  x  |  <  R 

+  [  sup  I  (r,  v,  w)  I  .  t*')  dr 

J  t  <  r  <  T  dx 

'  Iy|  <  2R 


where  €  [0.  1]  a.s.  for  every  n  €  N  are  given  by  the  mean  value  theorem  applied  a.*,  for  each 

ii.  I'siiig  the  estimates  of  Ocone  and  Pardoux  [15.  p.57j  ive  get  the  following  estimate 


-nip 

t  <  r  <  1 

|y|  <  2R 


da 

(h 


( r.  y. 


.')  I  <  sup 

0  <  r  <  T 

l>~,  <  2H 


Ji 

dx 


1st  <  2R 


X  1  l'x  +  Fu  u*  |  sup 

0  <  t  <  T 

|v|  <  2H 


|(~)'N)I  <  M  (!•(-)  (1+  4R")  +  (L(w)  (1+  1R’))") 


:=  fU)  (1  +  411-)- 


where  I. (u).  £(-.)  are  I.*  bounded  r.v.'s  .  We  use  again  the  notation  a(r,  x,  ut)  —  )" ^ ( x )  l(£r(x). 

ii*(r.  i-  (x)(.  Due  to  (a.  r).  F  is  bounded  and  so  is  Fx  +  Fu  u*  where  Fx.  F„  denote  Jacobians  with 
respect  to  the  first  and  respectively  second  variable.  M  denoting  their  common  bound,  u*  is  the 
Jacobian  of  u*(t.x)  with  respect  to  the  second  variable.  Applying  now  Gronwall  s  inequality  a.s.  we 
obtain 


A s(t/,  ut)  <  exp  (£(w)  (1  +  4R")"T). 


T  sup 
0  <  r  <  T 
f  x  |  <  R 


|  an(r,  x,  ut)  -  a(r,  x.  ut) 


s  <  r 


K 

R 


A  r 


K.n 

R 


Wo  show  next  that 


a"  (r,  x.  -')  —  a  (r.  x,  w) 

uniformly  in  (r.  x)  on  compact  subsets  of  [0.  T]  x  R^.  We  have  by  the  mean  value  theorem  for  I 


|  an  (r.  x,  u,)  -  a (r,  x.  w)  |  <  j  (^r^lx)  -  (^^(x)  |  |  F  |  +  |  (^^(x)  | 


x  |  Fx  +  Fuu*  |  I  €?(x)  -  «r(x)  |  <  «.p  |  (x)  (x)  |  M  +  M  b(-') 

ir,V* 

x  ( 1  +  R  ■ )  sup  |*?(x)-*T(x)  | 

0  <  T  <  I 
I  X  |  <  2R 


Using  now  the  uniform  convergences  in  probability  on  compacts 
£"(x)  -  fr(x) 


which  hold  under  (a.  c)  [U.  p.39].  [10.  p.5 1  f»]  wo  get  V.  R>0.  c.  b  3.  such  that  V  n  >  N ^ 


F  (  sup  |  a"(r.  x.  »)  •  atr.  x.  -■)  |  <  b)  >  P((L(^.)  <  L(  ) 

0  <  r  <  T 
|  x  j  <  2R 


k  (  sup  |  4?(x)  -  *T(x)  I  <  - - - 9-)  &  (  sup 

0<r<T  2ML  ( 1+R”)  0<r<T 

I x  |  <  2R  | x  |  <  2R 


L-  (  sup  |an(r.  x,  x,  .j)  |<  £  (Texp  (L((l  +  4R“)“T))'*))  >  1  -  p-  +  1 

(J  <  r  <  T  L  < 

|  x  |  <  2R 


-r  1  =  1  ■  3  forL'< 


6C' 


so  that  using  the  same  urgument  as  in  Shreve  and  Karatzas  [11,  p.  298] 


P(rj?  -  rRn)  -  P  (sup  I  U'C/Vl)  i  <  &  +  sup  |  V’ll  ('/)  I  for 
r;  €  K  t|£K 


t  <  s  <  A  rj^n)  >  1  -  </3 
Finally,  because 

lim  P  (sup  |  |  <  R-l  V.  s  €  [t.  T])  =  1 

R  — x  ,,£K 


and  so 


lim 

R-x 


P  (7-g  =  T)  =  1 


(i.c.  V.  i  >0  3,  Rfk  such  that  V.  H  >  P  (r^j.  =  T)  >  1  -  |  )  we  obtain  for  any  t  £  [0.  T]  and  for 

I\ 

any  compact  K  C  R<!  V,  t,  0  3.  :=  nJ^  such  that  V.  11  > 


P  (  sup  As( //.  <  f>)  >  P(AS  (»/.  ~)  <  />  for  t  <  s  <  T,  1;  €  K  ) 

t  <  s  <  T 
>/€  K 


>  P((AS  (7;.  ^)  <  /»  for  t  <  s  <  rj}  A  r£j'n  ,  77  €  K  )  k  (rj[  =  T)  &  (r^  <  rj},n)) 

>  I  -  f-  +  1  -  ^  +  1  -  I  -  2  =  i  < 


and  thus  we  managed  to  prove  in  pai'icular  for  s  =  T  (2.35).  We  can  now  prove  (2.3fi,J  . 
For  t  <  s  <  Tpj  A  rp’"  we  have 
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i 


Bs('>.  -/)  :  =  |  6  o  („■“)->  (V)  -  0  o  ^  o  (»/)  |  <  |  0K  |  |  f'j-  o  '(,,) 


-  s  |  o  t  '*  (r;)  |  <  M  (  |  f!j  °  (//>  -  o  (Vfs)_1  (»/)  I  +  UT  °  (V&)’1  (»;) 


-  f'j  o  <\l  (n)  I)  <  M  I  e\-  o  (v")-1  (v)  -  *T  o  (i'^)'1  (9)  I  +  M  |  |  hfs 


x  ( ^  { s )  1  (*J)  +  <*  -  7tns)  ijl  ('/*»  I  ( V’J\ )" 1  (f/)  •  *'t.s  ('/)  I  <  M  sup  |  (x) 

I  x  |  <  R 


-  (j  (x)  |  +  ML  (w)  (1  +  •1R-)  sup  |  (t/>"  )->  (,)  -  ^,-1  (r,)  | 

i  <  s  <  'i  [S 

n  €  k 


For  a  compact  K  C  R‘*.  V.  (.  ,x>0  3.  R,*x  and  Lf  =  @£-  and  :=  sucli  that  V.  n  >  : 


l’(Hs(»/.  -•)  <  b  for  t  <  s  <  r}}  f  rjj-".  »/  €  K  )  >  P  ((  sup  |£."r  (x)  -  *T(x)  | 

I  x  |  <  R*4 

<  k  (LM  <  L)  k  (  sup  lu  j’j-1  (»,)  -  tf'*  („)  |  < - t - ^)) 

t<s<T  2ML(  1+4R“) 

'/  €  K 


>  1  -  -=L-  +  1  .  L  +  t  .  i  .  •>  =  l  .  -1 
-  6  +  <>  1  6  -  _  1  3 

Hence  we  obtain 


P  (  sup  I3S( '/■  *v)  <  <“>)  >  P(iH,  (t/.  */)  <  (i-.  i  <s  <  Tp  A  7-{V".  »/  €  K)  and 
t  <  s  <  T  ‘  “  n.  iv 

»/€  K 

and  (rp  =  T)  and  (rj<  <  rjj"))  > 


(2.36')  is  I luis  proved.  Similarly  one  proves  (2.36)  by  applying  twice  the  mean  value  theorem,  by 
deriving  with  Cron  wall's  Lemma  the  estimate 


sup 

t  <  s  <T 
I  ol  <  2R 


Qf^)\  <  exp|  f(Jj(l+4R2)2T] 


with  C  (*;)  an  L* 


bounded  r.v.  as  before  and  by  using  the  convergence  result  for  each  t  <  T  : 


O'1  (x)  -  r0i  (x) 


uniformly  in  x  on  compact  sets  of  R 


d 


[3.  p.  66]). 


Proof  of  Proposition  2.1 


Consider  (2.12)  and  assume  (d)  and  u*(t.  x)  €  int  for  all  (t.  x)  g  [0.  T]  x  R"  as  in  (<•).  Then 
because 


\\  it.  //)  =  ft  o  Cj-  o  tj1  (<;) 


c/H  (t.  m  u>) 

(7u  V{t.  £,(tj))  rt''' 


and  because  H(t.  77,  w)  is  strictly  convex  on  the  compact  'll  for  all  (t.  ij)  a.s.  where 
H  (t.  r).  u.  ~)  =  ^  (t,  i,)  (h)  u)  -  i  8,x8i^t  (’?))  -  f“  Ut('))'  «*(»•  s,  ('/)))  »] 


we  have 


min 
u  g  1L 


H  ( t,  t).  u,  w)  =  H  (t,  ij.  u*(t,  ^(77)).  w)  a.s. 


1 

so  tlml  with  Aft,  77,  w)  defined  by  (2.10).  W(t,  17)  =  6  o  f  j  o  i/ij*  (77)  is  a  C~  solution  of  the  llamilton- 
Jacobi-Bellirian  equation  for  (P“)  a.s. 


i2.nn 


dw 

dt 


+  min„ 

u  € 


(t’  (S)1  {r,)  [f(^  (7;)’  U)‘  2  V‘(ft  (,))J 


■+■  A  r  (I.  q.  O.')  11}  =  0 

WIT.  rj)  =  6  o  (y(i/) 

As  solution  of  (2.40)  W(t.  q)  satisfies  tlie  optimality  principle  inequality 


W  (10.  X)  <  -  j  7JJ-  (r-  vr)  (Vr)  U  ((r  M  u*  (r>  tr^r))  u  ( r )  dlr 

l0 

+  W(t.  qt)  a.s. 


lor  any  u(t)  €  .T  where  qT  is  the  solution  of  (2.4)  starting  at  Iq  from  x.  We  also  have  under  our 
assumptions  (a.  0)  that  the  solution  «r  (2.4)  for  u  =  u’ft.  £((t)))  denoted  q*  .  exists,  is  unique,  non¬ 
exploding  and 

<IW(t.  //*)  _  (AV(t,  >/*)  m  ,  cfry£ 

(It  ”  fit  +  (h)  *’  dt 

=  ify  {I-  (,/*)  £((<  (,,*} • 11  * ((-  ^v.*)))  °  (»•  a-s- 

Mien  In  the  Verification  theorem  (see  Uleming  and  Rishel  (8.  p.  87])  u*(t.  £((r/))  is  o|>timal  for  (1’“) 
for  almost  all  *1  €  52  and  VV(t.  q)  is  the  value  function  of  these  control  problems  parametrized  In  w  €  52. 
It  remains  to  prove  that  V(t.  x)  :=  \V  (t,  £]*(x))  satisfies  (2.13).  Using  now  Lemmas  2.1  and  2.2  we 
have  for  each  t  €  [0.  T] 

Wty.tfJ*)-1  (x))£  W(t,  {J1  (X)) 

v"  (I.  X)  :=  w"(t.  (({')• 1  (x)  =  0o  (X) 
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J  f) 

uniformly  in  x  on  compacts  of  R.  As  (2.13)  has  a  unique  non  exploding  C"  solution  (because  of 
Kunita's  existence  and  urii<|ueness  theorem  for  SPDE's  from  Kunita  [12,  Section  VI]  whose  conditions 
are  met  here  (see  (a.c)).  we  must  have 


V  (t.  x)  :=  VV  (t.  rt‘  (x))  -  6  o  C;-1,-  (x) 


1  0 

because  of  the  uniqueness  of  the  limit  in  probability  so  that  W  (t,  ^  (x))  is  the  unique  C  solut  ion  of 
(2.13)  and  Proposition  2.1  is  proved. 


We  can  now  pass  to  the  proof  of  our  Lagrange  multiplier  theorem  for  antieipative  control. 

Proof  of  Theorem  2.1 

As  we  saw  in  Proposition  2.1  with  the  Lagrange  multiplier  process  AT(t,  r;.  -;)  defined  by  (2.1(1).  the 
optimal  control  for  the  stochastic  nonanticipative  control  problem  (2.6).  u*(t,  £{(r/))  is  optimal  for  the 
antieipative  optimal  control  problems  (P*“)  for  almost  all  ^  €  f2. 


Due  to  our  assumptions  An(t.  £j'(x))  defined  according  to  (2.10)  by 


A1  (t .  fj'fx))  :=  A1  (t.  x.  _)  =  •  -X  ll'  fu  (x.  u*(t.  X)) 


I.1  (d  P  x  dt)  -  integrable  lor  x  =  x,  the  solution  of  (0.1)  corresponding  to  some  u(  6  A 


1  i 

E  |  |  AT(t,  xt  u,)  |  dt  <  E  |  |  Vx  (t,  x,)  ||  fu  (xt.  u*(t.  xt)  |  dt  <  oc 


This  is  proved  in  the  following  way.  First  because  fu.  8  are  bounded;  Vx(t,  x)  =  0x((t  (x)) 


<K\ 1  ( -x ) 


—  and  x(=£(o»;(  we  see  that  via  the  estimates  for  flows  of  [15]  and  [3,  p.39.50]  : 


dC 1  >, 

sup  I  “3~(x)l  <  U^')(  1  +  |x|‘) 
t  <  T  <lx 


Vx  €  Rd  :  k(u.).l(w)  €  n  Lp(n)  w<-  have  that  the  integrability  of  the  Lagrange  multiplier  comes 

p  >  1 

down  to  showing 

T 

E  |  k(ui)[l  +  (lM(l  +  |7?t|2))2]dt  <  oo 
0 


aiul  thus  reduces  to  proving 


■>r 

sup  E  i),  <  x  Vi  >  I 

t  <  T 


I  Ins  follows  as  usually  from  the  existence  theorem  for  (1.4).  Consider  the  successive  approximation 


sequence 


J??=x0  '  v"  =  xo  +  (^rr1(t7i!'1)f(Cro^'1,uT)dr 

0 


We  assumed  before  without  restricting  generality  deterministic  initial  conditions,  so  that 


II  *>r  .j!' 

sup  i:|//"|-  =  |x0|-  <  x  . 

t  <  i 


By  induction  if 


sup  E|r;P'*|2  <  x  Vr  >  I 
t  <  T 


then  using  again  the  flow  growth  estimates  in  the  successive  approximation  sequence  denned  above  we 


have 


(2.41) 


t 

E|»?t  l~r  <  Ar  +  Br  +  Br|  E|»/“-1|2rdt-  <  oo 
0 

where 


with  cr  depending  on  r  only  and  M  the  bound  for  f.  Due  to  the  growth  estimates  and  the  a.s.  smooth 
flow  of  diffeomorphism  property  of  £((x)  the  coefficients  of  (1.4)  have  linear  growth  a.s.  and  are  locally 
Lipschitz  a.s.  so  that  the  existence  theorem  for  ordinary  differential  equations  can  be  applied  almost 
surely  to  (1.4)  :  the  successive  approximation  sequence  convergences  a.s.  to  the  unique  non-exploding 
solution  of  (1.4)  which  has  thus  finite  moments  due  to  (2.41)  and  moreover  by  (Irouwall  lemma 
applied  to 


<  Ar 


»r  + 


E|r/r|2  dr 


we  have  the  following  estimate 


E|r/,|-r  <  (  Ar  4-  Br)exp(  BrT;  Vt  £  [O.TJ.Vr  >  1 


We  show  next  (2. 11)  .  W  (t.  £['(x))  satisfies  (2.14)  and  because  of  u*  being  an  interior  optimum  (see 
(<  ))  for  (2.0)  (and  thus  ——  (t.  x)  £'x.  u*(t.  x ) )  =  0)  {'2.7')  can  be  written 


££  (t.  x)  +  ^  (t.  x)  [f[x.  U*(t.  x))  -  £  (x.  u*  (t.  X))  u*(t,  x)]  +  i  tr  (£^  ggT(x))  =  0 


ch 


0V* 

dx 


a2v* 


du 


dx~ 


V*(T.  x)  =  0(x) 


(2.42) 


We  average  the  integral  form  of  (2.14)  and  we  use  Lemma  6.2.6  and  Theorem  6.1.10  from  [12]  in  our 
particular  case  to  interchange  expectation  with  differentiation  and  integration  and  to  show  that  the 
stochastic  integral  has  zero  mean.  We  get  under  our  asumptions 


(2.42') 


X 


u*  (r,  x)]  dr  + 


(x))  dr 


where  we  denote  again  V(t,  x)  :=  \V(t,  (x)).  By  the  uniqueness  of  the  solution  of  (2.11)  (see 

[y.p.44] ) 


V*(t.x)  =  EV(i.x)  = 


(2.1:!) 


and  thus 


E  A 1  (i .  X)  =  E  A1  (t.  rt'  ( x ) )  =  I 


env(t.  ^ : 1  ( x ) )  m 

— sr —  £<-•  “"■*»> 


=  14  (1.  x)  £  (x.  -,*(«.  x)))  =  U 


We  have  proved  (2.11)  and  the  proof  of  Theorem  2.1  is  accomplished. 


Remark 


1  sing  the  convexity  assumption  ( 1)  the  H.1BSPDE  (2.14)  can  be  written 


dV'(t.  x)  +  mm^  {  Vx[f(x.  u)  -  £  (x.  u*(t.  x))  u]  +  \  tr  (Vxx  ggT(x))}  dt 


+  Vx  (t.  x)  g( x )  dw,  =  U 


V(T.  x)  =  0(x) 


where  Vx  :  =  .  V(t.  x)  is  tin  \mit  of  the  sequence  of  value  functions  for  the  sequence  of 


(deterministic)  control  problems  indexed  by  n  €  N  and  parametrized  by  w  €  0' 


*?  =  f(x".  U,)  +  g(x")  t,n(t.  -•) 

(Fu',n) 

T 

inf  [fl(x-p)  +  [  ( A"(t ,  xj\  ->)  )  Tu(t)dt] 
u€  A  *  J 
0 

where 


.  (An(t,  x.  w))T  =  -  v;  (t.*)$J(x.  u*(t.  x)) 

with  Vn(t.  x)  being  the  solution  of  the  sequence  of  random  HJB  PDE's  (2. 1 H).  I  sing  (2. Id)  we  have 
the  fundamental  result 


V*(t.x)  =  E  P-lim  V"(t.x)  =  EV(t,x) 

'  ti— x 

showing  how  the  second  order  parabolic  PDF  of  stochastic  dynamic  programming  results  by  taking  the 
limit  in  probability  of  the  sequence  of  deterministic  dynamic  programming  equations  (2.1*)  (which 
yields  the  HJB  SPDE  (2.1*1))  and  averaging.  This  is  the  quintessence  of  the  relation  between 
deterministic  and  stochastic  optimal  control  generalizing  to  dynamic  programming  equations  the 
VVong-Zakai  type  results  [20], [18]  proving  how  solutions  of  SDE's  appear  as  limits  of  sequences  of 
solutions  of  ordinary  differential  equations  .  Borrowing  the  title  of  Sussmann  [18]  ,we  can  say  that  the 
'gap  between  deterministic  and  stochastic”  optimal  control  is  filled  by  objects  like  HJB  SPDE  (2. I  I)  . 
The  price  to  pay  for  this  result  was  the  lengthy  convergence  argument  in  the  proof  of  Proposition  2.1  . 
Moreover  if  (gj(x),  gj ( x )]  =  0  for  i  <  ij  <  d  where 

^8;(x)  tfg.(x) 

(gj(x).  gj(x)]  =  -fa-  g,(x)  -  gj(x) 

(i.e.  gj  commute)  the  value  function  of  the  pathwise  optimal  control  problems  (2.2),  V(t.  x)=0  o  C,q-(x) 


is  contiiious  with  respect  to  the  Wiener  process  (due  to  the  characteristics  representation  this  follows 


from  the  continuity  w.r.t.  w(t,*v)  of  the  solution  of  SDE’s  proved  in  [18])  being  actually  the  "extension 


by  continuity”  to  C®  paths  (see  Davis  [.'])  of  the  value  function  V(t,  x)  of  the  problem 


X,  =  f  (  X,  ,  u, )  +  g(  X|  )  t  (t.  -') 


(P*) 


mf  [fl(x.f 

u  6 


)  + 


T 

}Ard. 

0 


-)  u(t)dt] 


where  t(t.  ui)  are 


C1 


noises  a.s.  and 


A'(t.  x.  *■)  =  -  Vx  (t.  x)  fjf-  (x.  n * ( t .  x)j 


Here  V  (t.  x)  is  regarded  for  each  *■  6  11  as  a  map  defined  on  the  function  space  C1  ( [0.  T].  R(l)  (i.e.  V 
is  mapping  the  path  space). 
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2.3  General  Lagrange  multiplier  formula  for  the  problems 
with  integral  cost 


If  we  consider  optimal  control  problems  with  integral  cost,  Theorem  2.1  can  be  generalized  as  follows  : 


Theorem  2.2 

Consider  the  following  family  of  optimal  control  problems  €  fi' 

'k  =  (j£)1  (r/t>  WV-  “tl'iE  «i  «i^t  Wt))l 

i=  1 

(P-)  »/„  =  *„ 


in  r 

u  G  *^3 


T  T 

[0  oCp('/|  )  +  1 1-  ('•  {,  °  ’If  V  <•'  +  J  A1  (t.  r)r  *•)  u,  dt] 


Assume  (a)  -  ( c )  and 

(d‘)  l.(t.  x.  u)  is  continous  in  t.  ("*  and  ( 'j*  in  (,x.  u).  convex  in  u  for  all  t.  x  and 

T 

{  £j  \0  0  ?T  0  +  }  IMr.{r°tr.  tj1  (Tj).  u*(r,  (r  o  o  t (r?)) 

t 

'  ih  (r‘  ('T  °  1  r  0  1  "*(r-  'T  0  1  r  °  ‘  i1  u*(r-  tr  0  *‘r  0  t  j1  ('/))  dr]} 

X  (^fV  (,,)  fuu  (*t  (1)  ’  U) 

has  characteristic  values  bounded  below  by  ■)>(!  for  all  (t,  T),  u)  €  [0,T]  x  R ^  x  'll  a.s. 

Define 

T 

AT  (t,  rj,  w)  :=  -  {  &  [6  o  {j  o  ti  , '(»))  +  J  (L  (  r.  £T  o  tfy  o  (»;),  u*(r,^rovroV'J1(f?))) 

'  (r'  **  0  *>  0  ^’t1^'*-  u*''r-  tr  0  +r  ®  I'J1  (r?)))u*(r,£r  o  0r  o  tf*1  (»>))  dr]} 


(2. 44) 


Then  u*(t,  ^,(/j))  is  optimal  for  (P"“)  a.s.  and 


E  \V(t.  ^(x))  =  V*(t.  x) 
E  AT(t.  ^(x).  -)  =  0 


where  \V(t.  i /)  is  the  value  function  of  (l*”*')  a.s 


Proof 

The  proof  is  identical  to  that  of  Throtem  2.1  except  that  this  time  the  integral  cost  term  changes  the 
characteristics  method  representation  for  \V(t.  r;)  as  follows  : 


Wd.  i/)  =  -t  o  c ;'(//)  =  T,  o  t  J-'i  ( v) 


where  ij  |  (t/)  is  given  as  before  by  (2.29)  and 


d* ,(»/) 

dt 


=  -I.(t.£t  o  c  ,  (r?).u*(t,^|  o  t ,(»,)))  +  SL  (t.  ((  o  ^t(v).u*(t.^oV't(r,))) 


du 


x  ti*(t.((ot  ,(>/)) 


1j(9)  =  6  o  £y(r/) 


2.4  Example  :  nonanticipative  LQG  Problem  solved  pathwise 


Consider  the  standard  nonanticipative  LQC  problem 


dx,  =  (Ax,  4  Bu,)  dt  4  C  dw. 


(p)  x0  =  x0  (2.45) 

T 

inf  E[|  (*f  Q  x,  4  Hu,)  dt  4  Xj  Fxj] 

0 


instead  of  solving  this  stochastic  optimal  control  problem  let  us  consider  the  family  of  deterministic 


problems 


d>7t 

dt 


=  A(//,  +  (V,)  4  Bu, 


-  *0 


(2.46) 


(P~) 


T 

inf  ^  { |  [('/|  +  Cvv,  )T  Q ( »/,  4-  (  w, )  +  u,1  Ru,J  dt  4  (ijj  4  Cwj)1  F 
0 


x  (r)j  4  Cwy)  4  A 1 1 1 .  I/,,  w)  u,  dt} 


We  need  to  determine  the  Lagrange  multiplier  A  (t,  ij t<  u>)  such  that  if  u*(t,  x)  is  optimal  for  (2.-15) 
then  ii * ( t .  t)  +  Cw,)  is  optimal  for  (2.46)  and  we  have 


V*(t.  x)  =  E  VV  (t.  x  -  Cw,) 


where  V*(t.  x)  is  the  value  function  for  (2.45)  and  W  (t,  tf)  is  the  value  function  of  (2.46).  The 
decomposition  of  the  solution  x,  is  here 


x,  =  I/,  +  Cw, 


dY  mm ^  {Vx(Ax  +  Bn )  +  x'Q  x  +  uTRu  +  AT  (t.  x,  w)  u  +  j  tr  { Vxx(’( ’ 1 )} 
x  dt  +  VjjC  <i w !  =  (l 


V(T,  x)  =  xrFx 


The  minimizer  u*(t.  x,  *»;)  is 

ii*(t.  x.  u«)  =  -  i  R'1  [BTV'i  +  A]  =  -  R_1BtS(  x  =  u*(t,  x) 
We  look  for  a  solution  of  (2.47)  of  the  form 


V(t.  x.  w)  =  xTS, x  +  2^'(«/i  x  +  -jt(w) 


which  we  plug  into  (2.47)  and  we  get 


xTSt  x  dt  ■+■  2d0]  x  +  d7(  +  [x  1  Q  x  +  j  tr  (C'TSC)  +  xT  S{  Ax  +  20]  Ax 
+  xT  AT  St  x  -  xTStBR'1BTST  x]  dt  +  (2  xTSt  C  +  20\  C)  dwt  =  0 

By  grouping  the  terms 


xT(S,  +  StA  +  AtS,  -  StBR*1BTSl  +  Q)  x  dt  +  xT(2d^t  +  2AT/?t  dt 
+  2S,(.'  dw,)  +  dy,  +  j  tr  |0TS,O  dt  +  2 0j  0  dw,  =  0 

we  see  that  we  must  have 

S,  +  S,  A  +  ATS,  -  S.BK-’b'S,  +  Q  =  0 
2d  0,  +  2  ATj,  dt  +  2  S,C  dw,  =  0 

d-i,  +  i  tr  (CTS,C)  dt  +  23]  C  dw,  =  0  (2.49) 

with  terminal  conditions 

ST  =  I 

•H  = 0 

V,  =  o  (2.50) 

because 

xtFx  —  xTSj  x  +  x  4* 

in  view  of  (2.48-2.50) 

A‘(t.  x.  w)  =  -  2 tf,rM  B 


independent  of  x  due  to  ^  =  B  =  const.  We  recovered  thus  the  results  from  Davis  [5).  If  we 
multiply  the  equation  for  dif,  in  (2.49)  by  Br  to  the  left  and  we  assume  BBT  nonsingular  we  obtain  a 


stochastic  differential  equation  for  A1  (and  not  a  stochastic  partial  differential  equation  HJB  SIMM:  a> 
in  the  general  nonlinear  case) 


dA[  =  -  BtAt(BBt)'1B  Xj  dt  -  BrStC  dwL 
A  j  =  0 

The  robust  equation  of  (2. 47)  for  W’(t.  r/),  where  W(t,  x  -  Cwt)  =  V(t,  x),  is  (see  (2.12)) 

Or-  (t,  r).  -j)  +  min  (A  (t/  +  Cw.)  +-  Bu)  +  A?(ui)  u  +  uTRu 

cn  dt)  1  l 

+  ('/  +  (V,)T  Q (>)  +  Cw,)}  =  0 
W(T.  n)  =  (n  +  (’«•-, -)1  !■(//  +  (  v  j  ) 

with  solution  VV(t.»/)  =  (j/+Cw()TSI(v+('wl)  -t-  23J(t)+Cw1)  +  ^ 

Remark 

Ihe  optimal  control  u’(t.x)  is  not  bounded  but  many  of  the  assumptions  of  our  results  can  be 

weakened:  Tl  need  not  be  conipat  t  as  we  use  here  the  fact  that  the  expression  to  be  minimized  in  M.1B 

<)£. 

IMM.  is  quadratic  in  u:  f  can  be  linear  in  x  as  £,(x)  =  x  +  t’wt  so  that  -^-(x)  =  1 , j  and  the  linear 
growth  in  x  of  a(t.x..j)  holds  a.s.  ensuring  the  nonexplosion  of  t',(t/)  in  (2.7);  the  characteristics  method 
works  with  9  having  Lipsehitz  contiiious  derivatives  so  in  particular  with  6  linear  etc. 
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3  :  Stochastic  anticipative  optimal  control 
and  almost  sure  (pathwise)  optimal  control 

We  consider  now  the  anticipative  stochastic  control  problem 

dx,  =7  (xt,  ut)dt  +  g(x,)odwt 

XU  =  *  0 

inf  E[<5(  x  r )] 
u  6 

(P°) 

x,  e  Rd,  g( x )  =  [gj(x) ;  ••• :  gp(x)] 
p 

f(x)  =  f ( x )  -  ^  X!  gi  (x)  gi(x) 

"  i=l  x 


to  which  we  associate  a  family  of  deterministic  optimal  control  problems  (P^'-')  jy  via  the  SHI. 
solution  decomposition  formula  (1.2)  \,(Xp)  =  or/,(xQ)  which  is  our  definition  ol  solution  ol  (1.1)  lor 
u,  €  -4  : 


where  Jk  is  the  class  of  measurable  functions  u:[0,T]— tU.'U  a  compact  in  Rnl.  The  relation  bet  ween 
(P°)and  (P°  \,€fi/  i» 


:>-1 


provided  again  that  the  infimum  on  the  right  is  attained  for  each  w  and  the  function  assigning  to  each 
u;  the  minimizer  of  P°,u;  js  measurable,  which  will  be  the  case  under  our  assumptions  as  we  use  a 
measurable  selection  .  This  shows  that  we  can  solve  (P°)  by  solving  (P°'u’)  for  €  Q  and  averaging 


the  cost  over  the  sample  space,  Q. 


We  make  the  following  assumptions 


(a') 


4 

f  is  in  (x,u)  (i.e.  with  bounded  mixed  derivatives  up  to  order  4)  and  bounded,  gj 
are  and  bounded  for  i~l . p.  6  is 


(<■')  the  nonanlicipative  stochastic  optimal  control  problem 
dx,  =  f(xt,u()dt  +  g(x,)  odw, 

x0  ~  x0 

inf  F[0(x(T))] 

tt  €  .N 

has  a  feedback  solution  u*(t.  \)  continuous  in  (t.  x),  ( in  x  so  that,  under  (a)  its 
value  function  is  the  C  '*  unique  solution  of 


nnn  < 
u  € 


OX* 

Ox 


(*.«)}+ 

0x~ 


V*(T.x)  =  6(x)  . 


(The  smoothness  assumptions  on  u*(t.x)  can  be  relaxed  if  we  impose  in  addition  the 
parabolicity  of  ggT  (8,p.  129]) 

(**)  Consider  Y(z,x)  =  tniri  {/, 1  f  (x,u)}  ,  z,x  €  R^.  Selection  lemmas  exist  (see  Bettes 

u  €  TL 

[2.  Lemma  5])  which  give  a  measurable  or  even  a  minimizer  0(z,x)=u^(z,x) 
such  that  Y(z,x)  =  t?  f  (x.o(z.x)).'i'he  implicit  function  theorem  can  be  used  to  find 


uniform 


conditions  for  smoother  minimizers  We  assume  here  that  (/>  is  c£  in  (x.z). 


Corresponding  to  each  member  of  the  family  of  problems  (3.2)  we  have  a  random  HJB  PDK  (i.e.  1»|)K 
with  random  coefficients)  for  the  value  function  W(t,/;)  :=  inf  [6  o  £(T,  t,  17))]: 

¥ + (&)''">  17  «.<»>•  “>} = 0  •»> 

W(T,rj)  =  Oo^(tj). 

Using  (e)  we  get  under  our  assumptions  a  minimizer  in  the  form 
u^t.v.^)  =  ^(t.,;)(^)'l(,?).  £((»/))  • 


which  substituted  in  (3.3)  yields 


w  +  fr  M  w>)  = « 

W(T.rj)  =  0  o  fT(v) 


I  sing  from  (1.2)  »/=£‘*(x)  we  get  the  expression  of  the  value  function  in  terms  of  x.  the  initial  value 
at  time  t  for  ( 1.1 ) 


V°(t.x)  :=  W(t.  f;1  (x)) 
u°(t,x.u>)  :=  u°(t 1  (x).w) 

and  it  will  be  shown  that  V®( t.x )  is  the  unique  global  solution  of  the  (backward)  stochastic  PDK 
(Hamilton- Jacobi- Bellman  stochastic  PDE) 

r  dv°(«-x)  +  ^(t.x)7  (x.o(^(t.x),x,)d«  -I-  ^(t,x)g(x)6  dw,  =  0  (3.5) 

l  V°(T.x)  =  0(x) 
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Tliis  solution  is  V^(t.x)  =  inf  9[x('l:t,x)]  and  is  expressed  by  the  stochastic  characteristics  method 

uw€-/ft> 

as  V®(t,x)  =  (x)  where 


<lr-,(x)  =  [f(^t(x),  «>(\t(x).  sst(x)))  +  \tr(x)fu(^t(x),«(\t(x),  <?t(x)))  0Y(\,(x).  y?,(x))]dt 


+  g(v?t(x))o  dwt;  -*?','(x)=x 


d \ , ( x )  =  —  [fx(v?t(x),  0(\|(x).  *t(x)))  +  fu(y5t(x),  <*>(.\t(x),  i,et(x)))^(xt(x),vst(x))]T  *t(x)dt 
d 

-  ]Cgi’x(^t(x))  Xt(x)°dw[  :  XT(*)  =  W  <3-7) 

i=l  ' 

d  i/. 

-^■(X)  =  \JM  fu(r,(x).  0(\,(x).^t(.\;!J<Px(\l(x)1  ,?,(x))xt(x) 

x )  =  0(x)  (3.8) 


:  dt: 


where  *u  =  's  11  x  111  matrix  of  components  fjj  =  0  =  0(w)  <P^  -  ^  l'lc-  "<>"  is  the 

Stratonovich  backward  differential  notation  and  (<^t(x),\t(x))  are  defined  as  in  (1.8), (1.9)  using  the 
flows  of  (3.<>).(3.7)  with  general  terminal  condition.  Making  the  notations 


F(^.\)  =  v1!  (y>.c>(  w)) 

fi(^.x)  =  x  rg(^) 


we  can  rewrite  (3.5)  as 


d\'()(t.x)  +  F(x.  ^-(t.x))  dt+G^x,  ^-(t.x)j  6  dwt  =  0 


f  dv”(t,x)  4 
l  V°(T.x )  =  0(x) 


and  (3.6)-(3.8 )  as 


drt(x)  =  k\(r*|{x).\,(x))dl  +  (•  X('r|(x).  \,(x))o  dw, 


d\t(x)  =  -  Fv?(y-,(x).\l(x))dl  -  ty^fx).  v,(x))6dw, 


(Til) 


(3.12) 


-g^U)  =  —  (F(^t(x),  \t(x))  -  Fv(v?,(x),  \t(x))\t(x)) 

^j(x)  =  X,  !?'[■( x)  =  0(\ ),  \T(x)  -  0x(x) 

(3.9)  is  a  nonlinear  SPDE  which  has  only  a  local  solution  for  t  6  ]r(w),T],(i.e.  down  to  a  stopping  time 
r(w)  ;  [10,  Chapter  6])  because  ^>,(x)  is  only  a  local  flow  of  diffeomorphisms  a.s.  up  to  a  stopping  time 
due  to  the  coupling  with  xt(x).  We  will  impose  conditions  ensuring  that  y?t(x)  *s  a  global  flow  of 
'  diffeomorphisms  a.s.  by  making  the  stochastic  Hamiltonian  system  (3.10)  -  (3.11)  admit  a  certain 

invariant  Lagrangian  submanifold  or  random  conservation  law  \,(x)=/f(t.^'l(\).-)  for  all 
(t.x)  €  [0.T]  x  a.s..  which  will  decouple  (3. 10)  from  (3.1 1 ). 

Equation  (3.9)  has  (3.4)  as  a  robust  random  PDE  (PDE  family  parametrized  by  ^6  51')  with 
characteristics  solution  VV(t.r;)  =  ot"(*(r/)  where 


(3.13) 
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(( w rwA  -  tti/( tv,)*,  -  -  pj(t 


/iT  =  0x(£T(r/))  (r?) 


(:u.T) 


(3.13") 


7T  =  0o£T(»,) 


Qf  1 

F  w(t.Vt.^t):=^T  (  -gff  Ml  («t(r).0(*T  (  ^)'‘(V)^t(V')))=F(v?,X)  |^w 

X=6T(d^t/^x)  —  1(V) 


aT/^Y1/ 


Wo  omit  q  from  the  notation  here,  writing  (i,,  <f>t t )  instead  of  ( Vt( J7)^, ( v)-"> | ( '/ ) )  defined  as  in 
(1.8)-n.l0)  using  the  flow  of  (3. 13). (3. 13')  with  general  terminal  condition.  We  also  omitted  for 
simplicity  the  variables  of  functions  in  the  second  and  third  equation.  Considering 


S2  -  ( 7GrJ'<‘'ii>T «.(»;]))  +  ( ( 


dx)  ut*» 


V’u('J)  =  r/ 


t  <  s  <  T 


we  have 


<■  ;*('/)  =  v\jUi) 
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3.1  Main  results  on  anticipative  control  and  on  the  cost  of 

perfect  information 


The  first  results  concern  the  optimal  control  and  optimal  cost  function  for  the  anticipative  stochastic 
control  problem  (P°)  solved  by  means  of  (P0,tJ)  with  Q'  as  in  Definition  1. 

LJ  t  W 

Theorem  3.1  Assume  (a'),  (e)  and 


(0  {F(v>.\).  \J-  0Xj(*)}  =° 


{G£(*\).  =  •» 

'  I 

for  (,r.\)  G  L={(y?,\ )  €  R-f*|\=0x(,r)},  C=l....,p.  i  =  1 d  where  { ■ .  • }  is  the  Poisson  bracket 


defined  by  {h(*v).k(*Y)}  =£ 


dk  dh_  \ 

dr  t  d\  -)- 


Then  u^(t.x,w):=u®(t.£^(x).^)=  arg  inf  E[0(x(T:t.x)))  =  <£(0x(x),x))  (i.e.  the  optimal  control  is  a 

u  €  -A 


feedback  control)  and 


V°(t.x.w)  =  inf  0(x(T:t.x))  =  0(x)+F (0.(?x(0))(T-t)  +  £  G£(O,0X(O)) 
uw  €  £=i 


x  [wj(T,^)  -  Wj(l.w)].V(t.x)  = 


inf  K[<9(x(T;t.x))]=  0(x)+F(O,0x(O))(T-t)  =  EV°(t,x.-,) 
it  €  -A 


J)  .  0 

where  Vu(l,x,^)  is  the  unique,  global  (’“  solution  of  HJBSPDE 


dV°(t,x)  +  ^min^{  ^(t.x)  f(x.u)Jdt  +  ^(t,x)g(x)o  dwt  =  0 


V°(T.x)  =  0(x) 


(3.15) 
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Remark  L  is  a  Lagrangiau  submanifold  as  the  symplectic  form  ^dXj  A is  null  on  MArnold  [I]). 

i=l 

The  next  theorem  characterizes  the  optimal  control  and  optimal  cost  function  of  (I’°)  in  another  case 
in  which  these  can  be  globally  defined  (i.e.,  for  t  6  [0,T],  x  6  R^).  In  this  case  the  optimal  control 
may  be  anticipative  and  V®(t.x,w)  is  not  separable  in  t  and  x  as  it  is  in  Theorem  3.1. 


Theorem  3.2  Assume  (a'),  (c)  and 

(f')  {Fx  (?,\).F(*.\)}=0  {FV.(*.X),  G£(^.X)}  =  0 
for  (y ,  \ )  €  R'd.  £=1 . p:  i=l . d 

(f"i  {Fx.(s.\).  \j  -  tfx  (^)}  =  0 

for  (^.\)  €  L 

I  hen  U(j(t.x.w)  =  ®(\t  or*(  (x).x)  and  Y^(t.x..v)  =  r<to^^(x)  is  the  unique,  global.  solution  of 
(3.1’*)  where 

d^fx)  =  F  v(0.t?x(0))clt  +  g(^,(>  ))<»  dwt.  y>y(x)  =  x  (3.l<>) 

and  t/,(x).  \,(x)  are  given  by  (3.7).  (3.8)  in  which  ^t(x)  is  the  flow  of  (3.16)  above. 


Kxample 

It  is  interesting  to  see  how  these  conditions  and  formulas  look  in  the  particular  rasa1  of  the 
deterministic  LQG  problem 


xt=Ax,  +  But  .  x0  =  xQ 


1 

u"e  jh,  ^  I  Q  xt  +  ut  R  ut*dl  +  4 f  *t  1 


We  have  the  HJB  PDIv  (dynamir  programming  equation  after  minimizing) 


f  +  Sr**  -  i  ®  BR‘‘BT(  *)' +  xTQx  • 0 


V(T,x)  =  xtMx. 


Condition  (f)  amounts  to  {F(*?,\).  \--2(M^)j}  =  0  on  the  Lagrangian  submanifold  L  =  €  R“f  |\- 

liMy=0},  where  F(^.\)  =  \TA^  — ^  \T  BR'*BT\+v>TQv?  is  the  Hamiltonian.  i=l,...,d  an<l  |Mr-)j 
denotes  the  i-th  component.  Aftei  calculations  we  get  that  (f)  is  in  fart 


Q  +  MA  +  A  rM  -  MBH  'B'Vl  =  0 


i.e.  M  must  satisfy  the  algebraic  Ricrati  matrix  equation  and  thus  is  a  stationary  solution  of  the 
Riccati  matrix  differential  equation.  We  get  V(t,x)  =  xTMx  (stationary  cost  function).  If  M.  solves 
instead  the  differential  Riccati  matrix  equation  (DRE) 


-  M,  =  Q  +  M,A  +  A  '.M,  -  .V1,BR'1B1Mt,  Mj  =  M 


V(t.x)  =  x  1  M,x 


because  DRE  for  Mt  amounts  to  the  condition  for  time  dependent  integrals  of  motion  for  stochastic 
Hamiltonian  systems  [3.  p.  231] 


^(Xi  -2(M|r-)j)  +  {F.  Xj  -  2(M|^);}  =  0  on  Lt  =  (*.x)  €  R2d|\  -  2M,r-=0} 
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whirl)  is  the  generalization  of  (f)(  {!  t,r.\),\  —  2M^}  is  the  vector  with  the  it h  componeni  being 
{F(V?<\)-\| -2(\V)j}  ). Indeed 


£(\  -2Mty>)  +  h^-^BR  JBTX  +  Q^.x  -2Mt^}  =  0 

on  L,  .This  can  be  seen  by  first  calculating  the  Poisson  bracket  and  differentiating  with  respect  to  time 
to  get 


-2M(^  +  ( -2Mt)(Ay-iBR'1BTv)  -  ATx-2Q^  =  0. 

We  then  substitute  \=2M.,r  and  factoi  out  2,?  to  get 

-  M{  =  MtA  +  A rM,  -  M,BR‘1BTMt  +  Q,  MT  =  M. 

We  state  now  the  generalizations  of  Theorem  3.1  obtained  by  imposing  conditions  for  the  existence  of 
time  varying  deterministic  and  random  .onsrrvation  laws  . 

Theorem  3.3 

O  1 

Assume  (a'),  (e)  and  assume  there  exist  .fj(t,^) .  Jj(t,,?)  which  are  in  -p  and  (  in  t  satisfying 

V(r-.V)  e  l,  =  {(r-\)  e  R-du-.f(t.r-)  =  0}  v.te[o.T] 

di- 

-^p(t.y?)  +  {F(yS,\),  \;  -  ^j(t.W')}  =  o  1=1 . d- 

(g)  =  o  i=l . d:  f=l . p 

.tyT-*>  = 


Then  u^(t,x)  =  o(^(t,x).x)  and  V®(t,x.»)  =  o^*{x)  is  the  unique,  global  solution  of  (3.5)  where 

d»?,(x)  =  Fx(>?( (x).  J(tlV?,(x))dl  +  t;x(^t(x).^(t,^t(x)))  6  dwt  (3.17) 

V?T(x)  =  x 
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(:U7‘) 


Ui/i  _ 

-gpx)  =  -  (F(s?,(x),  -Fx(^t(x),  /?(t,^t(x))))/?T(t,^t(x))) 

n-  j(x)  =  0(x) 


and 


W**'*)'  k(r,x,t)}  =  foreach  te(°-TJ- 


Remarks 


dv(t,^) 

l)  Assume  8-Al,<p)  =  — t - (i.e.  J-  are  gradients  of  some  y-,  for  t  fixed)  exist  satisfying  (g).  I  hen 

1  op  *  1 

V°(t,x.u;)  can  be  expressed  as 


V°(t.x,^)  =  ->(t,x)  +  f(t..j) 


d*(t.w)  =  *'(t)dt  +  £^(t)dwf  .  «T.->)  =  0 
f=l 


(3-1*) 


1  ^ 

where  d  is  the  backward  Ito  differential  notation  and  with  (  .Q 


given  by 


<'<■>=  -  K(> 


,2/. .  rM 

^(t)-  C,V‘-  (K-  / 


^-(tv)\ 
(V  / 


(;u!>) 


If  we  consider  stochastic  time-varying  prime  integrals  of  the  stochastic  characteristic  system  of  (3.">)  of 


the  form 
rtkl 


k . 


\,(x)  =  ^(t,^,(x),w)  we  gel  an  even  more  general  form  of  Theorem  3.2. 

■■^r-  -lkl  =  ki+  '  '  •  +  kn 

^n" 


We  denote  ojj  :  = 


Theorem  3.4 

Assume  (a')>  (e)  and  assume  there  exist  d  gradient  backward  random  fields  . 

.<(|(t.T?.w)(i.e.3,yj(t,^>,-v)  s.t.  J>j(t.^.w)  =  dyt(l.i?.u)/d<p)  which  are  in  *?.  V  t  €  (O.T]  a.s.  . 
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|D^j(t.x,u»)|  <  r(t.u>)  €  L*oc([O.T])  V  x  €  a.s.  for  |k|  <  3  (i.e.  for  some  processes  r(t,w)  almost  sure 
locally  integrable  on  [O.T]  )  and  have  differentials  given  by 


d£.(t,*?.u>)  =  /^(tv.-Jdt  J  6  dw^  ;  j=l . d 

J  £=1  J' 


satisfying  for  all  (^. \)  €  Lt(w)={(,r. \)  £  R"^|  \  —  /?(t,^.u/)=0}  and  for  all  w  in  some  J)"  with 
P(fi")=l  : 


^(t.yj.u,)  +  (F(y->.\).  \|  -  djO.y'.w)}  =  0 


.^f(t,y-w)  +  {Gf(y.\).  \|  -  ,ij(t.^.u;)}  =0 


i=l . d;  f=l . p. 


Jj(T.r-.-»)  =  flx(s) 


Then  it  ( l .x.^-)=0(  J(i.x.-).x))  is  I'-’j-  -adapted  and  V^(t.x,w)  =  -> ( t,x.-,)-t-  £  ((,«;).  £  (T.-.)  =  0  with 
£  (t.~)  given  by 


d£  ( t,-y)  =  £  *(t,w)dt  +  ^  £  jjj(t.w)  6  dw[ 
f=l 


*  +  i  ‘('-I  +  F(r-.  C-^0^)=  0  a.s. 

■)j|(t,y\U/)  +  £  £{t,w)  +  G^y.  =  0  a-s-  '=1 . P 


d->(t. *?,->)  =  7*(t.^.w)dt  +  ^  •)£((,*>,*/)  6  dwf 

f=l 


^  i  O'yi  0 

where  - ^ ‘(t,*?,*;),  •—  a.s. 

Oy  (t<y  ^ 

\’**(t.x.«;)  is  the  unique  V~  global  solution  of  (3.5)  and  it  can  be  represented  by  V^(t,x.w)  =  i/,  oy-J*(x) 


with  i/t(x),y>t(x)  given  by  (3.17'), (3. 17”)  where  instead  of  ^(t,^>t(x))  we  substitute  ,4(t.^(x  )..*>)  defined 
above. 

In  his  pioneering  work  on  stochastic  Hamiltonian  mechanics  [3],  Bismut  was  the  first  to  consider 
deterministic  conservation  laws  for  stochastic  Hamiltonian  systems  .  We  defined  here  a  generalization 
of  these  invariants  which  in  the  case  of  stochastic  systems  must  naturally  be  random  conservation  laws. 

We  can  now  state  the  results  concerning  the  “cost  of  perfect  information”:  the  difference  between  the 
nonanticipalive  optimal  cost  and  the  optimal  cost  when  anticipative  controls  are  allowed.  In  general 
we  have 


Theorem  3.5 

Assume  (a').(c')i(<?)  and  assume  that  for  all  x  6  y?t(x),  the  stochastic  flow  of  (3.6),  is  a  global  flow 

of  diffeomorphisrns  for  t  G  jO.T]  almost  surely  (which  is  true  under  the  assumptions  of  any  of  Theorems 
3.1  -  3.4).  Then 


T 


A(t.x)  =  V*(t.x)  -  F.V°(i.x)  = 
-f(x*  (s;t,x).e(\so^1(x*  (s:t. 


E[\s°v’;s1(x*(s;t.x))(f(x*  (s:t,x).  u*(s,x*  (s:t.x))) 
x)).x*  (s;t,x))))]ds  for  any  t  6  [O.T].  x  G  R‘‘ 


(3.22) 


where  x*(s;t.x)  is  the  solution  of 


dxs  =  f(xs,u*(s.xs))ds  +  g(::s)dws  .  x,  =  x  s  G  [t,T] 


In  the  case  when  the  system  of  stochastic  characteristics  (3.6)  -  (3.8)  admits  a  random  conservation  law 
.f(t.x.w).  which  is  the  case  of  Theorem  3.4,  we  have  in  particular  the  following  formula  for  the  cost  of 
information  on  the  future 


Corollary  3.1 


Ibtder  the  assumptions  of  Theorem  3.4  and  under  (c^)  we  have 
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Corrolary  3.2  Assume  that  either  (f)  or  (g)  hold  .  Then  A(t,x)=0 
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3.2  Proof  of  the  main  results 


Proof  of  Theorems  3.1-  3.4 

In  order  to  prove  Theorems  3.1  -  3.4  we  will  follow  the  same  method  as  in  §2.2.  We  consider  the 
pathwise  problems  (P^)^  g  q  and  random  PDE's  (3.3)  which  after  applying  the  selection  lemma  (see 
(e) )  become  (3.4).  Using  the  Verification  Theorem  [8,  p.  87]  and  the  characteristics  method  (3.13)  for 
W(t.  »/).  the  minimizer  u^(t, t;,^)  is  optimal  for  (P°'w)  and  W(t,ij)  is  the  value  function.  W(t ,//)  =  *, 
ofj'  ( r; )  (see  (3.13),  (3.14))  and 


where  i/}*  is  as  given  by  (1.4)  for  u,  =  u^(t. ?.*,). 

To  show  thai  \V(t,£*(x))  :=  Y°(t.x)  =  inf  o(x(T.t.x))  is  the  unique  global  solution  of  (il.'i)  we 

ii  w  €  -4b 

consider  linear  interpolation  approximations  {:  1,(t.w)}1)  ^  ^  (2.17)  of  w(t.o.)  in 


=  g(C')iJ,(l.*-) 


dt  " 


Corresponding  to  this  we  have  a  sequenc  e  of  random  PDE's 


Wn(T,t|)  =0o£"(T,) 


Then  by  ordinary  differential  calculus  VV!'(t,(^{’)‘ '(x))  :=  V®’n(t,x)  satisfies 


<9V°’n( 

dt 


1H1  +  2^  M?(«(«£W))  +  (,„  „„  if  ,  0 


ii  !■ 


68 


V°’n(T,x)  =  0(x) 


(3.24) 


As  in  §2.2  we  can  prove  the  following  convergence  results  in  probability  for  each  t  uniformly  .in  x  and  r/ 


V°'n(t.x)  -  V°(t,x) 


(3.23) 


\Vn(t ,»/)  —  W(t,v)  (3.2(i) 

VVd.f^x))*1)  -  Wft.^'lx)).  (3.27) 


where  V°  and  VV  are  the  solutions  of  ,3.5)  and  (3.4).  (3.25),  for  example,  can  be  shown  using  the 
stochastic  and  ordinary  characteristic-,  SDE  representations  to  reduce  (21)  to  show  the  convergence  of 
the  solutions  of  characteristics  SDK’s.  In  our  notation 


V°'"(>.x)  =  4lo(v?{')-1(x)  -  V  °(t,x)  =  ^o^fx). 


(3.2S) 


where  for  example  under  the  assumptions  of  Theorem  3.1.  as  we  will  see,  we  have 

d^n 

-jj^-(x)  =  Fx(^’(x).  6'x(rJI(x)))  +  g(*{,(x))&|\  *Jj.(x)  =  x  (3.29) 

i\t/n 

-jf-(x)  = 

x  */t,x^ = (3-;5o) 

drt(x)  =  Fx(r-,(x)0x(^((\)))dt+g(^t(x))6  dw,,  ^j(x)  =  x 
<WJs<x>  =  F\(>?ts(x)  3U p;*(x)))ds  +  g(/J(xl)odws,^}(x)  =  x 


«9 


(3.31) 


t  <  s  <  T: 


(3.32) 


^t1^)  =r'tT(x) 

Ii  follows  from  [3,p.39]  that  ^(x)  —  (x)  uniformly  in  probability  on  comparts  of  [O.T]  x  R1'  . 

j/.(x)  is  given  by  integrating  the  same  expression  as  in  (3-30)  but  with  *?t(x)  instead  of  ^[‘(x)  .  We 
used  the  forward  equation  (3.32)  to  get  the  inverse  of  the  backward  stochastic  flow  ^t(x).  It  follows 
now  from  VVn(t.(£”)"*(x))  =  V°’n(t.x).  (3.25).  (3.26),  (3.27)  and  the  uniqueness  of  the  solution  of  the 
stochastic  PDE  (3.5)  that  we  must  have 


vvii.rt1!*))  =  v°(t.x). 


For  all  this  reasoning  to  hold  we  need  to  ensure  that  both  (3.5)  and  (3.4)  have  a  global  solution  ( t In¬ 
former  holds  if  v?t(x)  is  a  global  flow  of  diffeomorphisms  a.s.).  The  following  lemmas  give  conditions 
for  the  existence  of  global  solutions  for  nonlinear  stochastic  PDE's  (3.9). 

Lemma  3.1 

Consider  the  backward  nonlinear  SPPi.  (3.9)  and  assume  (a'),  (e)  and  (f).  Then  (3.9)  has  a  unique 
global  solution 

V°(t.x.-,-)  =  0(x)  +rF((I.Pv(0))(T-t)  +  C;(O.0x(O))[w(T,w)-w(t.w)] 
i.e.  V^( t ,.\..v)  is  separable. 

Proof  (f)  is  equivalent  to  \,(x)  =  ('x(<^((x))  ,  t  G  [O.T]  where  xt(x),^,(x)  are  stochastic 
characteristics  given  by  (3.10),  (3.12).  Indeed  applying  Ko’s  backward  rule  [14.  p.255]  we  see  that 
(r-,(x).\,(x))  e  L  for  all  t  €  [O.T]  implies 

d\t(x)  -  ^xx(‘f’,(x))d17t('t)  =  -  F^(?tU),\t(x))dt 
-  ^(^(x).\t(x))6  dw,  -  0xx(^t(x))Fv(^t(x),\t(x))dt  -0xx(y>t(s)) 
x  Gx(V>l(x),xt(x))odwt  =  [F(*>,x),x-0x(<p)}|^_^(x)dt 

+  [Cl(^,\).  X-<?x(^)}|^=^(x)6dwt  =0  .  X"Xl(x) 

\  =  \t(x) 
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To  see  that  (f)  implies  (^t(x),\((x))  €  L  for  all  t  €  [O.T}  we  make  the  change  of  coordinate* 


T  : 


P  —Y 

\=\'W 


Indeed  T  is  a  change  of  coordinates  as 


d(p,\) 


‘d  °d 
*x*M  'd 


in  the  new  coordinates  (f)  is  equivalent  to  (f  ): 


T(I-)  =  L={(?.\  )€R2d|x=0} 


XF(Ti)lj-=(> 


where 


X  p(?.\  )  = 


<HP-\) 


—  t  n  y 


XF  (T  “  (p  )) 


*«<*>*)  = 


drcr-'^.x )) 

d(p<\) 


t=  1 . P  ;  i=l . d 


f  —  x  — —  n^-2-  ■ 
'Pdx,  Gr  xd<P  *d\  ' 


The  stochastic  characteristic  equations  in  the  new  coordinates  become  with  our  notations: 


tip  t(*) 

Xjr 

P 

(xot(x)-T  t(x))  dt  +  22 

X1  f 

Gl 

d*  t(x) 

X  F 

£=1 

x% 

g£ 

(p  t<x)’*  t<x))6dwtf 


(3.33) 


:=X  F  (v>t(x).\  t(x))  dt  +  Y  X  f(^t(x).\  t(x))°dwf  ;  y>T(x)=x.x  T(x)=0 

£=1 


and  (f  )  implies 

X  p(y?  t(x),0)  =0,  X2£(?t(x),0)  =0:  £=1 . p. 

G 

l  nder  the  assumptions  (a)  made  on  f,  g£  .6  (3.33)  has  for  all  x  €  Rd  a  unique  solution  and  this  is 
(7  “(x),"\  "(x)=0),  t  6  [0.T]:  where  ^“(x)  is  the  unique  solution  of 

P 

j(x)=X  p(^  “(x),0)dt  +  Y  x  2  f(?  r(x)'°)'  6  dwf  i  V  t(x)=0. 

£=1  G 


As  \  ‘(.x)=\t(x)  -0x(*?t(x))=U  ,  we  obtain  (y>,(x),\t(x))  €  L  for  all  t  €  [0,T]  . 


The  geometric  interpretation  of  (f)  is  that  in  every  point  of  the  Lagrangian  submanifold 
b={(v-.\)  £  R“a| \=t?x(rs))  the  drift  and  diffusion  Hamiltonian  vector  fields 


X 


F 


V  -  rC  (I  (.1  t) 

‘  ( iC  '  X<>r  '  *(>\ 


are  tangent  to  L  (we  use  the  vectorial  Poisson  bracket  notation  from  the  LQG  Example) 


-{F.\r*x.(*)}lL  =:  X,,(\r(9Xi(^))|L  =  0 

M  xrflx.(^)}lL  =  X^€<Xi  - ^x.(^))iL  =  0  5  i=1’-'d 

*  (l  1 


so  that  as  the  stochastic  characteristic  system  (3.10),  (3.11)  (seen  as  a  stochastic  Hamiltonian  system) 


starts  at  t=T  on  L  ( x-p( x )  —  0x(*2q’(x))  =  0  for  all  x  g  R^)  it  will  remain  there  for  all  0  <  t  <  'I'  a.s.. 
L  is  thus  an  a.s.  invariant  submanifold  for  (3.10).  (3.11).  As  a  consequence  >J,(x)  is  the  flow  of  (3.31) 
which  is  an  SDE  with  Lipschitz  coefficients  due  to  (a)  so  that  y?t(x)  is  a  (global)  flow  of 
diffoomorphisms  a.s.  for  t  6  [0,T][12].  Thus  V^(t.x)  =  cm^(x)  is  the  global  solution  of  (3.9).  But 
more  can  be  said  about  V^(t,x).  As 

^  (t,x)  =  Xt°y>;1(x)  =  ffx(x). 


V^(t.x)  is  separable 


V°(t,x)  =  0(x)  +  £(!.-•. 


To  find  £(t.w).  a  backward  random  field  with  the  differential 


d£(t,w)  =  f'ftjdt  +  ^^(t)dwf 
C=! 


(3.31) 


we  substitute  (3.34)  in  (3.9): 


[{’(0  +  F(x.0x(x))]dt  +  Yi  Kf(')  +  Ci£(x.tfx(x)))o  dwf  =  0. 
f=l 

Due  to  (f)  F(x.0x(x))  =  F(O.0X(O)).  G£(x.0x(x)l)  =  G£(O.0X(O))  so  that  £(t,w)=  F(O.0x(O)(T-t) 

P 

+  £g£(O,0x(O))[w€(T,w)-  w£(t.w)]. 
f=l 

Lemma  3.2  Consider  the  backward  nonlinear  SPDE  (3.9)  and  assume  (a),  (c').  (f)  and  (f").  I  hen 

f) 

(3.9)  has  a  unique  C“  global  solution 
V°(t,x)  =  j/jo^j^x) 
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with  y,'(x)  the  inverse  flow  of  (3.16)  and  given  by  (3.8)  in  which  ^(x)  of  (3.16)  is  substituted. 


Proof  (F)  and  (f")  imply  that  for  i=l . d. 

F\.(^t(x).\t(x))  =  Fx  (x,flx(x))  =  Fx.(0,ex(0)). 

i.e.  Fx(^t(x).\t(x))  are  d  prime  integrals  (conservation  laws)  (see  Arnold  [1],  Bismut  [3]).  As  a  result 
the  first  stochastic  characteristic  equation  is  an  SDE  (3.16)  (“decoupled”  from  \t(x))  and  >p((x)  is  a 
global  flow  of  diffeomorphisms  almost  surely  because  of  (a'). 

Generalizations  of  Lemmas  3.1  and  3.2  leading  to  Theorems  3.3  and  3.4  are  made  by  allowing  the 
conservation  law  to  be  lime  dependent  and  respectively  both  time  dependent  and  random  so  that  it  is 
a  random  field  having  a  certain  backward  differential.ln  this  way  (g)  and  (g'l  imply  that 
\,(x)=.f(t.,st(x))  and  respectively  \,tx)  =  J(t.^t(x)..*/)  (the  differential  of  the  random  field  being 
(3.20)  so  that  again  *5((x)  is  given  by  an  SDE  (“decoupled-  from  \t(x))  which  in  the  case  of  random 
Field  conservation  laws  has  random  coefficients.  In  this  case  the  global  diffeomorphic  property  of  y((x) 
follows  from  [I2.tj4.fi]  .  (a')  and  the  local  integrabilily  w.r.t.  time  a.s.  of  the  derivatives  in  x  of  the 
random  field  assumed  in  Theorem  3.4. 

We  need  a  result  connecting  the  existence  of  a  global  solution  for  (3.9)  to  the  existence  of  a  global 
solution  for  (3.4)  so  that  the  latter  is  implied  by  the  former.  We  will  prove  this  when  (f)  is 
assumed.  The  cases  when  any  of  the  assumptions  (f')-(f"),  (g)  or  (g')  are  made  are  treated  similarly. 

Proposition  3.1  Assume  (a'),  (e)  and  (f).  Then  (3.4)  has  a  unique  global  solution  given  by 
W(t.t;)  =  T  ,  o  f'i*(»;)  where  f  J 1  ( »/)  is  the  inverse  flow  of 

dg'  At))  _  1  - 

dt-  =  Fa(*tW'tto»’  iiv)))’ *  jit)  =  V  (3-35) 

and  ■>  ,  is  given  by  (3.13)  with  6  (=<?x  (t1  ,(tj))  and  4^=4'  ,(q)  substituted  in  the  equation  for 
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Proof  We  will  show  that  the  chararterisiic  sub-system  for  (3.4)  made  of  the  first  and  second 
equations  of  (3.13)  has  a  unique  global  solution  a.s.  given  by  (t/>t(»;).  liy)))-  first  due  to  the 

assumptions  on  f,g.  0  and  <s>  wc  see  that  (3.15)  has  unique  solution  and  (3.35)  has  unique  global 
solution  generating  a  global  flow  of  diffeontorphisms  a.s.  due  to  (t(x)  being  a.s.  flow  of  ( ''  global 
diffeomorphisnts  and  to  the  inequaiitytsee  Ocone  and  Pardoux  [15]) 


sup  (x)  <  £($)(l+|x|2)*  for  every  6  >  0 

t  <  T  Vc'x/ 


where  are  L.P- bounded  random  variables  for  p>  1.  (3.36)  plugged  in  the  expression  of  F  (see 

(3.13))  ensures  this  has  a.s.  linear  growth.  Showing  that  (3.13)  has  the  solution  (f ,(»;). 
d  ( =^x(  t  ,(r?))  amounts  to  checking  thai  (f)  implies  that  t  t=0x(t'!  t(rj))  satisfies  (we  omit  r/): 


cl  f  _  /()*  \  —  1  _  _  _ 

«)•*?' bx‘)  U  t)):='-F~;(t,ft.6l). 


f  |  =  ^x(^j('/)) 


that  is.  using  the  summation  convention. 


FJ _ t  ,F6-  I  =0 


t^t  J‘ 


We  first  check  the  terminal  condition 


6  =  6X  (<•  T(r?))  =  9x[r/) 


This  hoUls  because  (f)  is  equi valent  to 


o  =  hjgj£<*».  Xk-V^l,  “ 


V,*?  €  Rd;  £=!,.. .,p;  k=l . d  which  implies  that  for  all  1 }  €  R^,  t  €  [0,T]: 


d(,  <9£t 

Ox{ri)=Ox{tt(T}))-j^lT))  =  6X  (£t(j?))  a.s. 


This  can  be  seen  by  applying  Ito’s  rule 


"(%<<,<-<>©)>»  -  ©k."D  « a-f  +»Xj«,(i))(^-  sjeXS)ki(’<) 

0  d".f  =  (&ki""  •  d”i 


where  we  have  used 


d  =  57  «j£(W>  df)ki,r,)odw‘- 


WA,Ji  "xk  J 
As  a  consequence  we  also  have 


S(7)(S)  r  '(r,)  =  V^(,,)  ' 


We  differentiate  with  respect  to  »/• 


v^ii 1"'1 + v,;,4  ©kV(’'1 = 9w«'»Q)^ 


Remembering  the  definition  of  F(,r.\)  in  (3.9)  we  have  the  following  relations  Ijetweon  F  .  F^  aiul 

_  j  i  j 

F^where  ({,( t).AT  (^7)  (t ))  etc  and  Fv?k:=F^k^’x^  _  will> 

?  (t.w,v.<)=ft(v>),  X  (t ,U>X;6)  =  t1 


[«r  F-'k  (tstX‘ui  +  fxkin  ^  (^r)(k 


yf'A"w 
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which  when  introduced  in  (3.37)  yield  that  proving  the  proposition  now  amounts  to  showing  (we  omit 
arguments  of  functions): 


=0 


But  interchanging  the  summation  indices  in  (3.40)  j— i,  £— >k  and  evaluating  for  r/=c  (  we  gel 


M  (So, +v-vk  (&)'■)- 


lfft 


-  UlrM).  \£  ®X£U“)}iy-- ,v-  x  (rtx)fi^  t* 

where  -:=^(rt).  \(':=7  ,  (-'£)' ~  '  U  , ). 


\<>x) 

Due  to  (3.39)  and  f>  ,  =  $x(i  () 

\r  =  ^,1  =  W'O)  = 

for  all  I  €  [01]  so  that  (,?’.  \’)  €  l.  a.s.  for  all  t  €  [0.1]  and  (f)  implies 

{Fir'.x)-  =  o 

which  proves  the  characteristic  system  (3.13)  has  the  unique  global  solution  (v  t(v)-  ^  l(»/)=Sx(t-  t(t/))) 
and  thus  (3.4)  has  the  unique  global  solution  W((t,  r;)=7t o \l> ^(rj)  where  t/’i’lt/)  =  vtV('/)  with 
t  ’,V(o)  given  by  the  forward  random  equation 


d»"u<P)  _  v 
(Is 


=  ■«!('/))•  k  (r;i(o))i^f) 1  (t/’iift?))). 


t  •;}(»;)  =  v 


i  <  s  <  T 


Ci.11) 


Interpretation  of  the  result 

The  Hamiltonian  geometric  interpretation  ol  this  result  is  that  the  solution  decomposition  formula 
( 1.2)  leads  to  defining  a  canonical  point  transformation  (see  [3,  p.  339],  [1,  p.  239],  [16.  p.82]) 


P  =  ?,U') 


between  the  random  Hamiltonian  system 


(  v,  =  F  ^(t.c 0,)  :  fT=>/ 

1  =  -F^t.t ,. />,)  :  ^T=ex(vi  ('/))-^-(7) 

and  the  stochastic  Hamiltonian  system 

f  =  ^  «  y'J'  =  * 

{  d\,  =  -  F^(v?t.\,)dt  -  Cyl^.XtJodw,;  \T=0x(x) 


(;i.  12) 


(3.13) 


Being  canonical  the  transformation  preserves  Poisson  brackets 


{F(yM).  \  -flx(>-')}r-.\ 


which  can  be  shown  following  the  compulation?  from  the  classical  Hamiltonian  case  from  Huml  [17.  p. 
66-93]  for  our  case.  This  way  (3.43)  having  the  Lagrangian  invariant  submanifold  L  implies  that 
(3.42)  will  also  have  L  as  invariant  submanifold  via  (f).  One  can  proceed  like  this  using  the  theory  of 
canonical  transformations  for  stochastic  Hamiltonian  systems  of  Bismut  [3,  p.339]  to  develop  a  theory 
of  canonical  point  transformation  bt tween  stochastic  Hamiltonian  systems  and  the  robust  random 
Hamiltonian  systems  associated  with  them.  Proving  the  convergence  result  (3.26)  when  (f)  is  assumed 
means  in  the  light  of  Proposition  3.1  proving  that  for  each  t  6  [0,T] 


7* 


UIT1  (v)-v\'Oi ) 

uniformly  in  77  on  compacts  where 


<ll~cjt',,)  =  F^(e|*(v ?(»/)).  (V'?(»?))).  V’tC7?)  =  V- 

This  is  proved  in  the  same  way  its  (2.35).  The  random  characteristic  system  (3.1:J)  is  “decoupled"  ^ 
(>  [*=0x(i-  ")  ^  under  (f)  and  so  for  each  t  6  [0,T]  by  continuity 

_  p_ 

~  J‘(r/)  — 1  ,(r/)uniformly  in  77  on  compacts  thus  yielding  (3.26)  for  each  l  6  [O.T]: 

\Vn(t.r/)  =  ?  {“  O (f  [*)■'('/)  — VV(t,7 /)  =  T ,  ov;1('|) 

uniformly  in  77  on  compacts.  One  approaches  similarly  the  cases  when  any  of  the  assumptions  (f)  - 
(f"l.  (g)  or  (g')  are  made  In  ord<  r  to  “decouple"  the  stochastic  and  the  random  characteristic  systems 
and  to  ensure  the  global  existence  of  the  inverse  of  the  flows  of  characteristics. 


We  turn  now  to  the  “cost  of  informal io.T  issue. 

Proof  of  Theorem  3.5  and  ita  corollaries 

Averaging  (3.5)  and  interchanging  expectation  with  differentiation  and  integration  (we  use  again 
regularity  results  of  the  type  of  l.etntna  G.2.6  and  Theorem  6.1.10  from  [10]  in  our  particular  case)  we 


obtain 


(,-x>  +  1 1;  (KV«)(t.x)]f(x.u*(t.x))  -  [  |^(EV0)(t.x)]f(x,u*(l.x)) 
+  (t.x)f(x.0(V”(t.x).x)))  +  -}tr(|4  (EV°)(t,x)ggT(x))  =  0 


(•111) 


KV°(T,x)  =  0(x) 


where  we  added  and  subtracted  d/d\( EV®)(t.x)f(x,u*(t,x)). 
Subtract  (3.44)  from  (see  c')). 


(t<x)  +  (t,x)f(x,u*(t,x))  itr(  ggT(x))  =  0 

V*(T,x)  =  tf(x). 


We  obtain  the  PDE  for  the  cost  of  perl  eel  informal  ionfor  A(t,x)  =  V*(t,x) 


EV°t.x)  : 


7^  +  f(x.u*(t.x))  +  jtr(  0  ggT(x))  +  J;  (EV°)f(x,u*(t,x)) 

Acr.x)  =  o. 


We  obtain  (3.22)  by  representing  probabilistically  the  solution  of  (3.45)  and  using  the  stochastic 
characteristics  formula  for  V®(t,x): 


1 

A(t,x)=Et,x  |  E[Vx(s.x)^f(x,u4(s,x))  -  f(x,^(\'5(s,x),x))l  lx_x*^ds 
t 


Due  to  x*(s)  Ireing  independent  of  \x(s,x)  (the  former  is  past  adapted  while  the  latter  is  future 
adapted)  we  get  (3.22)  using  the  characteristics  representation  v!j|  (s,x)=xs  o  yrj'fx)  .  To  obtain  the 
particular  formula  from  the  Corollary  M  we  use  ^-(t.x)  =  0(t,x,u>)),  u®(t,x)  =  0(.f(t.x.-.).x).  (see 
Theorem  3.4).  We  prove  next  that  assumption  (f)  (i.e.  the  Lagrangi&n  submanifold  is  invariant  )  leads 
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to  zero  cost  of  information.  The  case  when  (g)  is  assumed  to  hold  (i.e.  existence  of  a  time  varying 
deterministic  conservation  law)  is  proved  similarly  using  generalized  Poisson  bracket  computations 
(time  is  included  as  additional  variable  )  instead  of  the  usual  Poisson  bracket.  As  we  have  seen  in 
Theorem  3.1  the  anticipative  optimal  control  is  nonanticipative  (feedback)  u^(t.x)=c>(0x(x).x)  and 

EV°(t,x)=0(x)+#x(O)^f(O,<p(0x(O),O))  -  ±gxg(0))(T  -  t) 

so  that  EVx=Vx=0x(x)  and  E  min^{Vxf(x,u)}=minc^{(EVx)f(x,u)}.  By  averaging  the  backward  Ito 
form  of  (3.15)  we  see  that  E\'x  satisfies  the  second  order  parabolic  PDE  of  stochastic  dynamic 
programming  (2.7’)  being  thus  equal  to  V*(t,x)  by  unicity  so  that  A(t,x)=0.  The  same  happens  when 
u^(t.x)=d(;5(t,x).x)  ,  EV’x=Vx=i?(t,x)  (  see  theorem  3.3  )  because  again  the  gradient  of  the  pathwise 
value  function  is  non  random  due  to  the  existence  of  a  deterministic  conservation  law  .  This  is  not  the 
case  when  a  random  conservation  law  exists  (Theorem  3.4)  or  when  (f),(f')  are  assumed  to  hold  and  in 
this  cases  A((.x)  ^  0  and  (he  optimal  control  is  anticipative. 
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3.3  Example;  Anticipative  LQG 


Consider  the  anticipative  LQG  problem  (first  considered  in  [5]  using  extension  by  continuity): 


dxt=(AX|+Bu,  )dt+Cdw( 

T 

inf  ^E[|  (x^Qxj+u^Ru^dt+XjFx-j-j  ;  Q  .  F  >0  ,  R  >0 

0 


(.'sing  the  extension  of  our  results  to  the  case  with  integral  cost  term  we  obtain  the  HJB  SPDF  of  LQG 
which  is  a  quadratic  nonlinear  SPDF 


dY°+[V°Ax  -  ( 1/4)V^BR" 1 B !( i1  +xTQx)dt+V®C odvv(=0 
V°|T.x)=x1Fx 


and  the  optimal  anticipative  conirol  in  selector  form  u0(t,x,u.')=  —  (l/2)R"^B‘V°.The  characteristics 
aref  using  the  lto  backward  differential  notation  (11.  |>.2*>5]): 


dv?t  =  [Av?,  -  (l/2)BR"*Br\t]dt  — Cdw,  ;  y*T=x 


-2Q^>t- ATxt  :  \T=  2Fx 


There  exists  a  random  conservation  law  \1(x)=2SI>r((x)+2^l(w)  where  S,  is  the  solution  ol  the 
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differential  matrix  Riceati  equation 


--3ji=StA+ATSt+Q-StBR'1BrSt  .  ST=F 


and 


diit(u))=/Sj(t,u.')dt-f^0(t.^')  odwt  ,  i?.p(w)=  0 


dS 


-  2-^y  -  2Jj(t,u;)+  {\  -  2S,y  -  2  V  rA^  -  \V TBR1BTY+yTQ*  }| 


€  Lt(^) 


=0 


-2ij(..U)+(»  -2V-  -2S,atC  €Ll(wr° 

L,M=t(,>.\><=«2,ll  i=2S,v+25,l-j  ) 


We  tan  comment  again  t hat  although  the  coefficients  of  the  SPDE  and  SDR  are  not  bounded  with 
bounded  derivatives  the  existence  of  an  affine  conservation  law  leading  to  an  affine  first  characteristic 
equation  .decoupled  from  the  second  one.  yields  a  global  solution  .It  is  important  to  point  here  that 
such  a  Hamiltonian  mechanics  point  of  view  already  led  to  interesting  re-derivations  of  the  Kalman 
filter  by  Bensoussan, Bismut, Mitter  [3,  p.356].The  value  function  and  the  optimal  anticipative  control 
are  : 


V°(t.x)=x',S,x+2j|U-)x+  .  u°(t.x.-;)=  -  R-1Bt(S,x+/?,M) 

d^t=-(Ar-  S, BR‘ ' B r)<y,dt  -  S, Cdw,  , 
d7t=/&trBR'1BTi?ldt-L,d[Cdwt  ,  7t=0 


It  is  interesting  to  see  that  Riccati  matrix  differential  equation  appears  in  the  conservation  law 
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formula.  The  cost  of  perfect  information  is  (  see  also  [5]  )  : 

T 

A(t,x)  =  tr(  l!tBR*1BT)dt 
t 

where  U.  is  the  symmetric  nonnegative  definite  solution  of  the  Lyapunov  equation 


=  (A7  -  StBR‘1Br)Ut  +  Ut(A-BR'1BTSt)  +  StCCTSj  :  l 
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3.4  Example  :  a  nonlinear  anticipative  control  problem 


Consider  next  the  scalar  nonlinear  anticipative  optimal  control  problem  : 


/cosx.+2  sinx,  \  .  , 

dx,= - 7 - 1- - - — ^  +  u,  dt  +  ■  J  ,  dwt 

1  V  2  2{cosxt+2)3  COSXt+2  1 


inf  [  7  j  u^dt  4-  sinxj  +  2xj 


Using  the  obvious  extension  of  our  results  to  problems  with  integral  costs  we  get  the  nonlinear  HJB 
SPDK 


dv»  +(  .nin^  V$u  +  J.*  )  +V$  a^d,  +  vSgJL^dd.,  =  0 

V®(T,x)  =  sinx  --  2x 


which  for  the  minimizer  u"(L,x,«;)  =  -  becomes 


IV*  +(v» *¥*  +  vJasbs***, - 11 


We  can  check  that  (f)  holds  : 


.  cos^+2  i  •>  .in  o 

h  — 7 - f\  •  \  -‘■0V--,}Il=  0 


' -cos^ -2>h.  =  0 


for  (y.\)€  L  =  {(*?.\)€R2d|  \-cosf-2  =  0}  and  thus  the  Lagrangian  submanifold  L  is  invariant 
for  the  stochastic  characteristics  of  this  HJB  SPDE 


dy,(  =(^!rtll_Xt)d(  +_i_6dw,  ;^T  =  x 
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—  cosx  —  2  . 


d\t  =  X,  — j-*  dt  -  \t - ^  6  dwt  ;  xT  =  cosx  +  2 

(COSy',+2)- 

i.c.  \(  =  cos,;t  +2  V  t  6  [0,T]  a.s.  The  optimal  control  is  feedback  u^(x)  = 

V^(t,x,w)  =  sinx  +  2x  -f  Wj  —  wt  and  EV^(t,x)  =  V*(t,x)  ,  A(t,x)=0  (  zero  cost  of  perfect 
information)  as  sinx  +  2x  is  also  the  solution  of  the  parabolic  PDE  of  nonanticipative  optimal  control 


av*  , 

at  + 


mm  < 
u  6  H1 


-t 


iu2j  +  v.  (cos^+J  + 


2(cosx  +  2)J 


V* 

*  XX 


(cosx+2)' 


=  0 


V*(T.x)  =  cosx  +  2 
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